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PREFACE 

The  research  effort  represented  by  this  report  was  funded 
by  the  Commandant  of  the  Marine  Corps. 

The  objective  of  the  research  effort  was  to  evaluate 
feedback  mechanisms  for  MCCRES .   Hopefully,  the  reader  will 
judge  that  the  objective  has  been  satisfied.   In  our  opinion 
it  has  been  exceeded  due  largely  to  the  formal  and  informal 
support  offered  us  by  the  Marine  Corps  officers. 
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EXECUTIVE  SUMMARY 

I .  Purpose  of  the  Study 

The  study  was  conducted  at  the  direction  of  the 
Readiness  Branch  Headquarters  Marine  Corps  (Code  POR) . 
The  study  was  conducted  by  Major  T.  P.  Finnerty,  USMC, 
Professor  K.  J.  Euske  and  LtCol  J.  F.  Mullane,  Jr.,  USMC 
of  the  Administrative  Sciences  Department  at  the  Naval 
Postgraduate  School,  Monterey,  California.   The  study  was 
used  to  answer  the  following  research  questions: 

(1)  After  MCCRES ,  what  information  should  be  fed  back 
to  the  evaluated  Unit  Commander? 

(2)  In  what  form  should  the  evaluation  format  be? 

(3)  What  channel (s)  should  be  used  to  provide  the 
feedback  to  the  evaluated  unit? 

II .  Methodology  of  the  Study 

The  study  was  conducted  during  the  period  July- 
September  19  83.   The  goal  of  the  study  was  to  provide 
answers  to  the  primary  research  questions  as  well  as  to 
three  secondary  research  questions  which  will  be  dis- 
cussed later.   Both  published  documents  and  interviews 
were  used  to  develop  the  answers  to  the  research  questions 
The  literature  reviewed  for  purposes  of  this  study  in- 
cluded research  literature  in  the  areas  of  management 
control  and  evaluation  and  DoD  documents.   Interviews 
were  conducted  with  a  total  of  37  Marine  Corps  officers. 
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These  officers  ranged  in  rank"  from  Captain  to  Colonel. 
Thirty-six  of  the  officers  had  significant  experience 
with  MCCRES .   Through  a  detailed  weighting  system 
emphasis  was  given  to  the  responses  of  those  officers 
who  held  key  billets  during  a  MCCRES.   The  most  heavily 
weighted  were  those  officers  who  had  served  as  Battalion/ 
Squadron  Commanders,  Senior  Evaluators  and  Division/Wing 
MCCRES  Officers.   The  results  of  the  study  provided  the 
data  from  which  a  comprehensive  feedback  model  was  de- 
signed which  incorporated  answers  to  the  three  primary 
research  questions  on  feedback. 
Ill .   Answers  to  Primary  Research  Questions 

The  three  primary  research  questions  were: 

(1)  After  MCCRES,  what  information  should  be  fedback 
to  the  evaluated  unit  commander? 

(2)  In  what  form  should  the  evaluation  be? 

(3)  What  channel (s)  should  be  used  to  provide  the 
feedback  to  the  evaluated  unit? 

The  overall  goal  of  the  MCCRES  is  not  only  to  pro- 
vide a  measure  of  unit  readiness,  but  to  provide  infor- 
mation to  the  organization  to  improve  readiness  by 
applying  resources  to  areas  identified  as  deficient  by 
MCCRES.   It  was  evident  throughout  the  study  that  the 
unit  commander  faced  many  constraints:   training,  time, 
material,  equipment,  logistics,  and  personnel.   There- 
fore the  information  from  MCCRES  should  be  provided  in 
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such  a  way  as  to  allow  for  better  allocation  of  these 
scarce  resources  to  the  unit  to  improve  readiness. 

In  both  theoretical  and  applied  models  of  evaluation, 
information  and  feedback  play  an  important  role  in  both 
performing  and  using  the  results  of  an  evaluation.   In 
general  all  models  of  evaluation  have  common  desirable 
characteristics  such  as:   brevity,  clarity,  timeliness, 
interim  products  and  reports,  and  responsiveness  of 
feedback.   Therefore  the  MCCRES  should  also  incorporate 
these  common  characteristics  of  a  good  evaluation  system. 

Based  upon  literature  review  and  the  results  of 
the  field  study  interviews,  a  model  for  MCCRES  feedback 
was  developed.   The  model  is  presented  at  the  end  of  the 
summary.   Each  of  the  eight  major  elements  of  the  model 
that  affect  the  evaluated  unit  are  explained  below. 

1.  Pre-MCCRES  Briefing 

A  detailed  orientation  several  weeks  prior  to  MCCRE 
to  provide  a  firm  foundation  for  all  key  personnel. 
The  orientation  should  provide  a  detailed  explanation 
of  appropriate  volumes  of  MCO  3501.2.   Explanation  of 
the  details  of  each  MPS  and  procedures  for  the  evaluator 
allows  the  unit  to  maximize  the  potential  from  the 
MCCRE  feedback. 

2 .  Real-Time   Feedback 

Regular  verbal  feedback  throughout  the  exercise  by 
evaluators  at  each  level.   At  any  logical  break  in  the 
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scenario  detailed  evaluations  of  the  most  recent  events 
should  be  provided  by  the  evaluators.   This  provides 
data  while  the  event  is  fresh  in  everyone's  mind. 

3 .  Evaluator  Worksheets 

The  Evaluator  Worksheets  should  be  provided  directly 
to  the  evaluated  element  shortly  after  conclusion  of  the 
exercise.   This  provides  an  interim  evaluation  report 
and  allows  for  maximum  feed-back  to  the  specific  element 
evaluated.   Worksheets  of  the  evaluators  would  be 
especially  valuable  to  subunits  and  attachments  who 
often  receive  only  minimal  feedback  under  current 
procedures . 

4 .  Two-way   Debrief 

At  the  end  of  exercise,  twelve  to  twenty-four  hours 
after  the  end  of  the  formal  MCCRES ,  a  detailed  debrief 
should  be  held  for  evaluators  and  key  players  from  the 
unit  evaluated.   Opportunities  should  be  given  for  two- 
way  discussion  to  resolve  any  confusion  or  disagreement 
particularly  regarding  subjective  areas.   The  individuals 
from  the  evaluated  unit  should  leave  with  not  only  a  clear 
understanding  of  the  deficiencies  but  also  with  a  clear 
understanding  of  what  caused  the  "No"  evaluations  and 
suggestions  for  improvement.   Since  the  commander  of  the 
unit  evaluated  is  responsible  for  the  employment  of  the 
feedback,  he  should  have  the  flexibility  to  tailor  the 
debrief  to  his  leadership  style.   The  commander  may  de- 
sire to  have  only  a  few  selected  evaluators  and  officers 
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present  or  the  unit  commander  may  desire  all  evaluators 
and  unit  officers  to  be  present. 

5.  Computerized  MCCRES  Printout 

— 

The  computerized  MCCRES  printout  with  percentile 
scores  and  detailed  amplifying  remarks  should  be  pro- 
vided to  the  unit  in  a  timely  manner.   It  does  a  unit 
little  good  to  get  the  detailed  results  when  there  is  no 
time  to  correct  deficiencies.   What  is  timely?   This  can 
vary  depending  on  deployment  schedules,  but  getting  a 
copy  of  the  results  in  the  hands  of  the  evaluated  unit 
so  that  action  can  be  taken  is  essential.   It  is  inex- 
cusable and  unprofessional  to  allow  staffing  to  delay 
this  critical  information.   Delays  also  put  the  unit 
further  away  from  the  MCCRES  and  final  written  results 
tend  to  become  historical  documents  vice  a  map  for  future 
training. 

6 .  Comparative  Percentile  Results 

The  data  needed  to  compare  units  within  a  division  or 
wing  is  currently  available  with  limited  research  within 
the  division  or  wing  files.   Additionally  most  division 
or  wing  headquarters  are  well  aware  of  how  their  internal 
unit  scores  compare  with  other  like  units.   Usage  of  the 
general  MCCRES  data  bank  to  provide  comparative  scores, 
medians,  and  standard  deviations  should  be  provided  to 
major  headquarters  staff,  training,  and  doctrine  commands 
for  the  limited  purpose  of  planning  future  resource 
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allocation  and  to  develop  training  doctrine.   These  com- 
parative results  should  be  limited  to  Section/MPS/Task 
scores  and  should  not  include  any  sort  of  comparative 
final  score  for  the  individual  units  tested.   Eecause 
of  the  many  variations  in  evaluation  conditions  that 
exist,  providing  any  sort  of  overall  scores  that  could 
be  used  to  compare  individual  units  tested  could  well  be 
misleading  and  have  a  negative  impact.   It  is  further 
recommended  that  these  comparative  results  be  directed 
only  at  division/wing  level  or  higher  commands  to  reduce 
the  chance  of  any  type  of  Battalion/Squadron  unit 
comparisons . 

An  example  of  comparative  percentile  results  that 
could  be  provided  to  division/wing  level  for  use  in 
planning  future  training  is  presented  in  -Figure  1. 

PAST  SIX  MONTHS  MCCRES  RESULTS 

Division/    Overall 
Wing     Marine  Co; 

Section   2.0    Operations  Performed 

MPS      2. A    Actions  by  Marines 

Task      2.A.1   Discipline 


Figure  1.   Sample  of  Comparative  Summary  Percentile 
Results 


7 .   Follow-up  Report 

Currently  no  follow-up  evaluation  is  made  to  confirm 
that  the  deficiencies  have  been  corrected.   Some  type 
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of  final  check  is  needed.   This  check  would  best  be 
accomplished  with  little  formality.   Since  the  unit  com- 
mander has  the  responsibility  for  application  of  MCCRES 
feedback  to  improve  unit  readiness,  it  is  best  left  to 
the  unit  commander  to  do  the  final  check  which  certifies 
that  deficiencies  have  been  corrected.   Resource  reallo- 
cation from  outside  the  evaluated  unit  likewise  should 
be  certified  at  the  appropriate  level .   The  key  issue 
here  is  that  the  unit  must  be  deployed  at  the  highest 
level  of  combat  readiness  and  verification  of  correction 
of  MCCRES  deficiencies  at  least  on  an  informal  basis  is 
essential.   This  follow-up  should  be  made  as  simply  as 
possible.   The  unit  commander  can  appropriately  perform 
this  certification  based  on  his  own  experience  and  judg- 
ment .   A  formal  follow-up  inspection  is  neither  necessary 
nor  desirable. 
8  .   Trend  Analysis 

To  gain  a  higher  level  of  across-the-board  readiness, 
a  trend  analysis  would  be  of  value  to  all  units.   The 
MCCRES  data  base  represents  a  resource  that  could  be 
used  to  keep  the  Marine  Corps  informed  of  its  overall 
strengths  and  weaknesses.   Special  emphasis  should  be 
made  on  getting  these  results  to  training  and  doctrine 
organizations  throughout  the  Marine  Corps. 
IV.    Answers  to  Secondary  Research  Questions 

The  study  also  provided  answers  to  secondary  research 
questions.   The  three  secondary  research  questions  were: 
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(1)  What  is  the  validity  of  comparing  MCCRES  results 
given  under  differing  conditions? 

(2)  Does  MCCRES  contribute  to  effective  training? 

(3)  Is  there  the  available  time  and  follow-up  to 
ensure  correction  of  deficiencies  identified 
during  MCCRES? 

A  first  response  to  the  question  of  the  validity  of  com- 
paring MCCRES  results  given  under  differing  conditions 
might  be  to  say  that  such  comparisons  can  only  be  made 
if  complete  standardization  is  achieved.   Although  com- 
plete standardization  for  purposes  of  evaluation  may 
seem  to  be  desirable,  this  is  not  feasible.   Since  various 
Marine  Corps  units  (east  coast,  west  coast,  overseas) 
prepare  for  different  missions,  it  is  desirable  to  simu- 
late their  expected  combat  employment  through  the  use  of 
different  scenarios,  terrain,  and  MPS's.   Additionally 
the  Marine  Corps  has  other  exercises,  such  as  the  Com- 
bined Arms  Exercise  (CAX's) ,  which  are  used  to  evaluate 
under  standard  scenario,  terrain,  and  evaluators,  but 
they  have  a  more  limited  purpose  than  MCCRES.   Addition- 
ally, because  of  the  broader  purposes  of  MCCRES,  complete 
standardization  is  probably  not  desirable.   For  its  in- 
tended purpose,  MCCRES  provides  a  relatively  high  degree 
of  objectivity  when  compared  with  some  other  evaluation 
systems,  such  as  the  performance  evaluation  system  used 
for  the  promotion  of  Marine  officers,  and  the  results 
are  likely  to  be  useful,  at  least  for  generic  comparisons 
and  for  identifying  trends. 


In  response  to  the  question  of  MCCRES * s  contribution 
to  effective  training,  it  is  evident  from  the  study  data 
that  MCCRES  has  the  potential  to,  and  currently  does, 
improve  unit  training  in  preparing  for  a  MCCRES.   Many 
units  train  to  MCCRES  standards  and  go  so  far  as  to  use 
actual  MCCRES  Section/MPS/Task/Requirements  to  structure 
their  unit  training.   Therefore  as  the  Individual  Train- 
ing Standards  (ITS)  are  developed  for  all  units,  it  is 
essential  that  the  training  and  evaluation  development 
move  in  the  same  direction. 

The  third  question  concerns  time  availability  to 
correct  MCCRES  deficiencies.   A  comment  often  brought  up 
during  the  study  was  that  insufficient  time  to  correct 
operational  deficiencies  exists  between  the  time  a  MCCRES 
was  taken  and  the  time  of  deployment.   This  is  a  criti- 
cal factor  after  a  MCCRES.   It  does  little  good  to  simply 
identify  deficiencies  of  a  unit.   A  set  aside  period  of 
time  must  be  provided  in  training  schedules  and  used  for 
remedial  training  in  areas  identified  as  weak.   The  time 
needed  must  be  judged  on  a  case  by  case  basis  and  correc- 
tive training  applied  accordingly.   The  researchers  were 
often  told  during  the  study  that  MCCRES  is  just  one  more 
hurdle  to  be  jumped  on  the  road  to  deployment  and  that 
once  it  is  past  the  unit  seldom  looks  back  to  correct 
operational  deficiencies.   No  doubt  more  time  would  allow 
for  more  training  in  deficient  areas.   However,  the 
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question  of  follow-up  is  also  critical.   As  stated  above 
at  least  an  informal  re-validation  of  deficient  areas 
is  needed  to  ensure  the  effectiveness  of  the  MCCRES . 

The  study  provided  detailed  answers  to  the  questions 
posed.   The  feedback  to  the  unit  is  the  key  element  to 
making  the  MCCRES  of  value  to  the  unit  evaluated.   The 
purpose  of  MCCRES  is  to  provide  timely  and  accurate 
evaluation  of  the  force  readiness.   Key  to  improving 
readiness  is  identification  of  strengths  and  weaknesses, 
and  providing  timely  and  thorough  feedback  to  the  evaluate 
unit  and  its  chain  of  command.   The  more  timely  and 
thorough  the  feedback,  the  higher  the  likelihood  of  im- 
proved future  readiness.   The  MCCRES  is  currently  viewed 
as  a  sound,  valuable  evaluation  system  by  those  who  have 
been  involved  in  it.   However,  as  this  study  has  demon- 
strated, the  feedback  to  units  can  and  should  be  improved. 
V.     Study  Recommendations 

The  feedback  model  developed  and  displayed  at  the 

end  of  the  summary  should  be  incorporated  as  the  minimum 

acceptable  procedure  for  providing  feedback  from  any 

MCCRES.   To  accomplish  this,  the  eight  steps  in  the 

feedback  model  should  be  followed  during  each  evaluation. 

The  following  information  should  be  routinely  provided 

as  feedback: 

Pre-MCCRES  briefing — to  key  MCCRES  players  from 
the  evaluated  unit 

Real  Time  Feedback — distributed  to  all  those  evaluate< 
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Two-way  Debrief  at  end  of  exercise — tailored  to 
needs  of  the  evaluated  unit  commander  to 
ensure  full  understanding  of  evaluation  results 

Computerized  MCCRES  Printout — provided  in  a 

timely  manner  to  allow  new  resource  allocation 
prior  to  deployment 

Comparative  Percentile  Results — to  help  direct 
unit  training  and  for  incorporation  by  indi- 
vidual training  and  doctrine  commands 

Follow-up  Report — done  informally  by  the  unit 
commander  to  ensure  closure  of  the  feedback 
loop  by  the  correction  of  identified  deficiencies 

Trend  Analysis — to  support  future  training  as  well 
as  changes  in  doctrine. 

Implementation  of  these  recommendations  would  suit 
the  stated  needs  of  fleet  units  as  well  as  serve  the 
intended  purpose  of  MCCRES  of  providing  an  evaluation 
of  unit  readiness  to  higher  headquarters. 
VI .    Summary 

The  purpose  of  the  study  was  to  determine  the  most 
appropriate  and  effective  manner  in  which  to  provide 
feedback  to  the  unit  commander  from  a  MCCRES.   To  this 
end  research  was  conducted  into  the  area  of  existing 
systems  which  measure  or  evaluate  readiness. 

The  study  was  approached  from  a  multi-disciplinary 
background  with  a  detailed  emphasis  on  the  economic 
question  of  how  better  distribution  could  be  made  of 
the  scarce  resources  of  manpower,  equipment,  and  training 
time  based  on  the  results  of  MCCRES.   To  answer  the  ques- 
tions a  detailed  field  study  was  conducted  in  which 
interviews  were  completed  with  37  Marine  officers  of 
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rank  from  Captain  to  Colonel  who  have  had  involvement 
with  MCCRES  as  key  billet  holders.   Additionally,  re- 
search literature  in  the  areas  of  management  control 
and  evaluation  of  much  of  the  original  documentation 
that  laid  the  groundwork  for  the  development  of  MCCRES 
was  reviewed  along  with  interviews  with  some  of  the 
original  developers  of  the  system. 

The  results  of  the  study  are  displayed  as  an  eight 
step  feedback  model  which  is  based  on  previous  research 
in  the  fields  of  management  control  and  evaluation,  as 
well  as  the  results  of  the  field  study.  These  results 
are  communicated  to  the  reader  as  a  model  presented  at 
the  end  of  the  summay  which  is  comprised  of  the  follow- 
ing elements: 

(1)  Pre-MCCRES  Briefing 

(2)  Real  Time  Feedback 

(3)  Evaluator  Worksheets 

(4)  Two-way  Debrief  at  the  end  of  Exercise 

(5)  Computerized  MCCRES  Printout 

(6)  Comparative  Percentile  Results 

(7)  Follow-up  Report 

(8)  Trend  Analysis 

The  incorporation  of  this  model  as  standard  MCCRES 
feedback  procedure  will  significantly  enhance  the  value 
of  the  results  to  the  evaluated  unit  and  will  improve 
the  understanding  of  the  resource  allocation  needs  at 
all  levels. 
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I.   INTRODUCTION 

The  Military  has  long  recognized  the  need  for  forces  to 
be  prepared  for  combat.   The  United  States  Marine  Corps  has 
historically  been  considered  this  country's  force  in  readi- 
ness.  A  means  to  evaluate  and  measure  readiness,  called  the 
Marine  Corps  Combat  Readiness  Evaluation  System  (MCCRES) ,  was 
implemented  by  the  Service  in  19  77.   MCCRES  was  designed  to 
provide  readiness  information  to  all  levels  of  command.   How- 
ever, the  system's  ultimate  value  lies  in  the  communication  of 
information  to  the  unit  evaluated  and  to  the  higher  level  com- 
mands that  can  support  improved  allocation  of  resources  and, 
ultimately,  a  higher  level  of  readiness.   This  research  views 
the  MCCRES  from  an  interdisciplinary  perspective  and  specifically 
looks  at  the  economic  consideration  of  improving  the  allocation 
of  scarce  resources  of  manpower,  equipment  and  training  time. 

A.  PURPOSE 

The  purpose  of  this  study  is  to  look  specifically  at  the 
feedback  resulting  from  a  MCCRES  evaluation  and  to  consider 
all  of  the  MCCRES  information  available  to  the  Marine  Corps 
and  to  determine  how  that  information  could  be  used  to  improve 
combat  readiness . 

B.  SCOPE  AND  METHODOLOGY 

Initially  a  detailed  literature  search  was  conducted  into 
the  areas  of  management  control  and  evaluation.   Following 


the  detailed  literature  search,  the  background  and  develop- 
ment of  MCCRES  was  reviewed.   The  MCCRES  is  applied  through- 
out the  Marine  Corps  to  evaluate  ground  and  air  combat,  as 
well  as  combat  service  support  units.   Most  commonly  the 
MCCRES  is  applied  at  the  infantry  battalion  and  aviation 
squadron  level.   In  total  over  500  MCCRES' s  have  been  conduc- 
ted since  implementation  in  1977.   Therefore,  a  sizeable  body 
of  Marine  Corps  Officers  still  on  active  duty  have  served  in 
key  billets  during  MCCRES  evaluations.   To  capitalize  upon 
their  experience,  a  field  study  was  conducted  which  included 
interviews  with  the  initial  developers  of  MCCRES  (both  Marine 
and  civilian  personnel) .   In  addition  key  questions  on  MCCRES 
feedback  were  posed  to  those  who  had  been  involved  at  all 
levels  of  MCCRES  but  with  special  emphasis  on  those  who  had 
served  as  unit  commanders,  senior  evaluators ,  or  division  or 
squadron  MCCRES  officers.   Based  on  this  research  the  answers 
to  questions  posed  by  this  study  were  answered. 

C.   ANALYSIS 

The  information  obtained  was  subjected  to  two  levels  of 
analysis.   First  specific  responses  to  the  field  study  were 
analyzed  through  content  analysis.   Next  an  analysis  of 
general  responses  to  the  data  collected  was  conducted  iden- 
tifying MCCRES  elements  which  were  repeatedly  presented  by 
those  interviewed.   Finally,  a  summary  of  unsolicited  comments 
were  compiled  to  provide  completeness  to  the  study  and  are 
suggested  as  areas  for  further  research. 


D.   RESULTS  AND  CONCLUSIONS 

The  results  and  conclusions  are  provided  in  the  form  of 
a  feedback  model  consisting  of  the  following  eight  elements: 

1.  Pre-MCCRES  Brief 

2.  Realtime  Feedback 

3.  Evaluator  Worksheets 

4 .  Two-way  Debrief 

5 .  Computerized  MCCRES  Printout 

6.  Comparative  Percentile  Results 

7.  Follow-up  Report 

8.  Trend  Analysis 

Overall  the  MCCRES  is  viewed  as  a  valuable  and  necessary 
measure  of  unit  readiness.   However,  to  maximize  the  effec- 
tiveness of  MCCRES,  timely  distribution  of  results  must  be  made 
to  the  unit  evaluated  and  all  others  directly  in  the  chain  of 
command.   The  timely  distribution  of  results  will  allow  for 
improved  resource  allocation.   Specifically,  the  distribution 
of  results  has  a  high  value  for  improving  the  use  of  limited 
training  time  and  the  development  of  doctrine  that  supports 
the  effective  and  efficient  use  of  Marine  Corps  resources. 
Thus,  regular  distribution  of  generic  results  should  be  pro- 
vided to  all  training  and  doctrine  commands  throughout  the 
Marine  Corps  to  improve  areas  consistently  identified  as 
deficient. 


II.   BACKGROUND 

The  Marine  Corps  is  interested  in  assessing  Unit  Combat 
Readiness.   Readiness  is  generally  viewed  as  the  peacetime 
level  of  preparedness  to  go  into  combat.   The  Marine  Corps 
Combat  Readiness  Evaluation  System  (MCCRES)  is  designed  to 
measure  units '  level  of  readiness  through  simulated  combat 
conditions  as  determined  by  specific  criteria  set  forth  in  the 
form  of  mission  performance  standards,  tasks  and  requirements, 
and  as  judged  by  experienced,  expert  evaluators  [Ref .  1] . 
The  readiness  demonstrated  is  actually  a  complex  set  of  re- 
sults based  on  the  effective  allocation  of  resources  by  the 
service  and  the  unit  commander  [Ref.  1] .   Management  control 
encompasses  all  management  actions  taken  by  the  service  and 
the  unit's  commander  to  bring  the  unit  to  peak  efficiency. 
The  evaluation  system  employed  is  a  detailed  effort  to  quan- 
titatively determine  the  success  of  that  resource  allocation. 
This  chapter  explores  the  key  concept  of  readiness  as  it  has 
historically  evolved.   It  discusses  the  basics  of  MCCRES  and 
lays  groundwork  for  the  detailed  research  that  is  directed  at 
answering  key  questions  involving  feedback  of  MCCRES  data  to 
the  unit  commander  for  the  specific  purpose  of  upgrading 
deficiencies . 

A.   MARINE  CORPS  COMBAT  READINESS  EVALUATION  SYSTEM  (MCCRES) 
Historically  the  Marine  Corps  has  been  called  upon  as  a 
ready  force  to  meet  contingencies  on  short  notice  world-wide. 


The  specific  Marine  Corps  Missions  are  detailed  in  the  National 
Security  Act  of  194  7.   The  unique  air-ground  combat  team  organi- 
zation in  which  each  unit  ranging  in  size  from  the  Marine 
Amphibious  Unit  (MAU)  to  Marine  Amphibious  Force  (MAF)  is  task 
organized  with  Marine  Corps  Air,  Ground,  Combat  Support,  and 
Combat  Service  Support  elements  making  the  organization  uniquely 
suited  to  respond  rapidly  to  U.S.  global  needs.   Recent  inci- 
dents such  as  the  1979  Iranian  Crisis  and  the  19  82  Lebanese 
Crisis  have  placed  current  emphasis  on  this  historic  role. 
1 .   Purpose 

The  operational  capabilities  of  combat  units  describe 
their  abilities  to  function  for  their  intended  purpose.   It 
is,  therefore,  necessary  that  some  means  be  used  to  estimate 
their  ability  to  perform  their  intended  mission  short  of  actual 
combat.   In  19  76  the  Commandant  of  the  Marine  Corps  (CMC) 
identified  the  need  to  develop  an  operational  readiness 
evaluation  program.   In  his  Posture  Statement  to  Congress 
for  19  78,  General  Louis  H.  Wilson,  Commandant  of  the  Marine 
Corps,  made  the  following  statement  describing  a  need 
for: 

...an  improved  readiness  evaluation  system  to,  provide 
a  timely  and  accurate  evaluation  of  the  readiness  of  the 
Fleet  Marine  Forces,  including  reserve  units,  to  accom- 
plish assigned  missions.   [Ref .  1:  p.  I-A-l] 

The  Marine  Corps  began  using  the  system  in  July,  1978.   Since 
that  time  the  system  has  been  used  to  test  all  Marine  Corps 
Infantry  Units,  Fixed  Wing  Squadrons,  Rotary  Wing  and  Obser- 
vation Squadrons,  Combat  Support  and  Combat  Service  Support 


elements,  both  regular  and  reserve,  ~at  least  once  every  24 

months.   In  his  FY-84  Posture  Statement  to  Congress,  General 

Robert  H.  Barrow  (then Commandant  of  the  Marine  Corps)  made 

the  following  comment  about  MCCRES . 

A  key  element  in  the  readiness  equation  continues  to 
be  operational  performance  standards.   Even  before 
these  units  deploy,  we  evaluate  their  capabilities 
to  accomplish  their  assigned  missions  through  the 
rigorous  Marine  Corps  Combat  Readiness  Evaluation 
System.   The  System  objectively  evaluates  the  combat 
readiness  of  all  active  and  reserve  FMF  (Fleet  Marine 
Force)  Units.   [Ref.  2:  p.  47] 

The  goal  of  MCCRES  is  to  test  the  combat  readiness  of 
Marine  Corps  units.   These  tests  have  been  developed  to  evalu- 
ate the  five  types  of  units  mentioned  previously.   There  are 
difficulties  in  applying  a  standard  such  as  MCCRES.   The  real 
question  that  needs  to  be  answered  for  the  organization  is: 
"Can  the  unit  do  the  job?"  [Ref.  3:  p.  1] .   Since  the  real 
answer  can  only  be  obtained  under  actual  combat  conditions, 
the  Marine  Corps  must  substitute  simulated  combat  for  actual 
combat.   Thus  the  question  that  is  answered  is:   "How  close 
is  the  execution  to  doctrine?  [Ref.  3:  p.  1] .   Thus  for  MCCRES, 
the  Marine  Corps  tests  adherence  to  doctrine,  under  simulated 
combat  conditions  as  a  proxy  for  the  real  question. 
2 .   Scoring/Results 

The  guidance  for  MCCRES  is  defined  in  detail  by  Marine 
Corps  Order  (MCO)  3501.2,  Volumes  I-X.   This  provides  details 
on  how  the  MCCRES  is  applied  to  each  type  of  unit.   The 
evaluation  is  broken  down  into  ten  (10)  categories  that  are 
mutually  exclusive  and  collectively  exhaustive. 


1.  Reporting  to  higher  level  of  command 

2.  Preparing  for  operations 

3.  Communicating  (including  communications  SOP) 

4.  Performing  as  Marines  (e.g.  discipline,  dispersion, 
camouflage,  concealment,  using  weapons) 

5.  Delivering  supporting  fire 

6 .  Planning  of  operations 

7.  Conforming  to  doctrine 

8.  Executing  operations 

9.  Providing  combat  service  support  (including  medical 
support) 

10.   Supervising  required  actions  by  individual  Marines. 

Each  category  corresponds  to  a  vital  aspect  of  the  unit's 

performance  [Ref.  3] . 

For  an  Infantry  Battalion  the  evaluation  is  divided 
into  19  Mission  Performance  Standards,  14  5  Tasks,  and  79  3  Re- 
quirements.  Each  Battalion  is  not  tested  on  all  MPS,  Tasks, 
and  Requirements  [Refs.  1,  4 J  .   The  final  result  is  a  "Combat 
Ready/Not  Combat  Ready"  for  the  unit  evaluated  based  on  the 
overall  judgment  of  the  senior  evaluator.   Results  are  also 
provided  as  percentile  scores  for  the  Section,  MPS,  and  Tasks 

The  format  for  information  provided  is  as  follows: 

Infantry  Unit 

Operations  Performed  %  Score 

Actions  by  Marines  %  Score 

Discipline  %  Score 

Self  Discipline  %  Score* 

*Note:   Only  a  yes/no  is  given  for  a  Requirement,  thus 

10  0%  =  Yes  and  0%  =  No  for  each  Requirement  Graded 


Section 

2.0 

MPS 

2. A 

Task 

2.A.1 

Requirement 

2. A. 1.1 

For  example,  the  Infantry  Evaluation  is  subdivided 
into  four  Sections:   Section  2. A  identifies  standards  applica- 
ble to  all  evaluations.   It  is  divided  into  three  MPS,  the 
first  of  which  is  Action  by  Marines.   'Actions  by  Marines'  is 
subdivided  into  13  Tasks — the  first,  'Tasks,  Discipline'  is 
subdivided  into  nine  Requirements.   The  first  of  these  re- 
quirements is  'Self-Discipline.'   Requirements  are  evaluated 
as:   Yes/No/Not  Applicable.   Section,  MPS,  Tasks,  and  Require- 
ments' overall  evaluations  are  each  given  a  weighted  percentile 
score  based  on  the  number  of  Yes's  for  Requirements  under  the 
respective  category. 
3.   Procedure 

The  MCCRES  is  divided  for  simplicity  into  five  phases. 
Phase  Cne  requires  determining  the  appropriate  MPS  to  be 
evaluated  based  on  Marine  Corps  Order  3501.2,  Vol.  I,  and  the 
appropriate  volume  that  applies  to  the  unit  being  evaluated — 
for  instance,  air  or  combat-service  support.   Phase  Two  con- 
sists of  briefing  evaluators  [Refs .  5,6J.   Phase  Three  is  the 
actual  evaluation  under  simulated  combat.   During  this  phase 
the  evaluators  actually  make  the  Yes/No  evaluations  of  the 
requirements.   They  are  guided  by  detailed  Key  Indicators 
(KI)  which  assist  the  evaluator  in  making  the  appropriate 
choice  [Ref.  4J .   Phase  Four  consists  of  compiling  the  evalua- 
tion results  and  determining  the  percentile  scores  as  well  as 
the  "Combat  Ready/Not  Combat  Ready"  determination  by  the 
senior  evaluator. 

Officially,  there  is  no  relationship  between  the  numeri- 
cal score  for  the  Battalion  and  the  Combat  Ready/Not 
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Combat  Ready  rating.   It  is  possible,  for  example,  that 
one  battalion  with  an  overall  numerical  score  of  50 
could  be  judged  Combat  Ready,  while  another  having  a 
numerical  score  of  75  could  be  judged  Not  Combat  Ready. 
[Ref.  7:  p.  40] 

Phase  Five,  the  final  phase,  is  the  debrief  of  the  evaluated 

unit  by  the  evaluators  as  well  as  the  detailed  written  report 

(computer  printout)  which  is  provided  to  the  evaluated  unit. 

Other  reports  are  forwarded  to  Headquarters  Marine  Corps 

(CMC) — an  initial  message  report  within  10  days  and  the 

detailed  report  within  30  working  days  [Ref.  1], 

B .   READINESS 

At  the  center  of  any  Readiness  Evaluation  System  is  under- 
standing what  is  meant  by  the  key  concept  of  "readiness." 
The  inherent  difficulty  in  describing  this  term  makes  any 
evaluation,  measure,  or  model  of  readiness  all  the  more 
complex.   Thus  we  must  explore  the  meaning  of  "readiness." 

1 .   Definition 

Usage  of  the  term  "readiness"  has  tended  to  change 

even  at  the  highest  levels  within  the  Department  of  Defense 

(DoD) .   In  FY  1977,  then  Secretary  of  Defense,  Donald  H. 

Rumsfeld,  used  the  following  definition  of  readiness: 

"Readiness"  is  a  concept  that  integrates  the  diverse 
factors  that  affect  the  ability  to  deploy,  engage,  and 
sustain  effective  combat  forces.   It  starts  with  the 
overall  ability  and  proficiency  of  U.S.  fighting  men... 
An  equally  important  determinant  of  overall  readiness 
is  the  availability,  capability,  and  condition  of  the 
force's  fighting  equipment.   [Ref.  8:  p.  2] 

In  FY  19  7  8  the  same  Secretary  of  Defense  is  quoted 

as  follows: 


"Readiness"  refers  to  the  capability  to  respond  adequately 
to  diverse  situations  and  to  sustain  that  response  as 
long  as  necessary.   The  "Readiness"  of  Defense  Combat 
Forces  depends  on  a  myriad  of  diverse  and  often  inter- 
related factors.   [Ref.  8:  p.  2] 


Under  the  current  administration  the  concept  has  taken 

a  more  standardized  definition.   In  an  address  to  Congress  on 

February  8,  1982,  and  January  31,  1983,  Secretary  of 

Defense  Weinberger  used  the  following  definition: 

Readiness  is  the  ability  of  forces,  units,  weapons 
systems,  or  equipment  to  deliver  the  outputs  for  which 
they  were  designed  (including  the  ability  to  deploy 
and  employ  without  unacceptable  delays) .   It  depends  on 
having  the  required  quantities  of  equipment  in  the  hands 
of  the  units  on  a  day-to-day  basis  and  on  having  the 
required  number  of  adequately  trained  people  assigned 
with  the  necessary  mix  of  grades  and  experience  level 
and  to  ensure  that  people  and  machines  can  work  to- 
gether.  [Ref.  9:  p.  1-28] 

This  definition  includes  training  (individual  and  unit), 
material,  equipment,  logistics,  and  personnel  all  as  part 
of  the  readiness  definition. 

For  the  purpose  of  continuity  and  brevity,  the  follow- 
ing definition  of  readiness  is  used  on  a  day-to-day  basis  by 
the  Marine  Corps  and  is  the  accepted  definition  from  JCS , 
Publication  6,  used  for  the  DoD  Unit  Reporting  System 
(UNITREP) . 

Readiness:   ability  of  forces,  units,  weapons  systems, 
or  equipments  to  deliver  outputs  for  which  they  were 
designed  (includes  the  ability  to  deploy  and  employ 
without  unacceptable  delays) .   [Ref.  10:  p.  1] 

It  is  further  amplified  as  follows: 

Readiness  is  essentially  a  measure  of  pre-D-Day  status 
(extending  at  most  into  initial  combat  operations)  while 
sustainability  is  a  Post-D-Day  measure.   Hence,  we 
often  speak  of  peace  time  readiness,  but  combat 
sustainability .   [Ref.  10:  p.  1] 
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For  consistency  this  definition  of  readiness,  as  amplified, 
is  used  throughout  this  study,  since  it  provides  the  cur- 
rently accepted  DoD  definition.   In  the  literature  the  terms 
'effectiveness'  and  ' sustainability '  are  often  interchanged 
to  indicate  Post-D-Day  combat  capabilities.   Because  of  its 
broader  implications  the  term  effectiveness  is  used  herein 
to  mean  Post-D-Day  status.   Hereafter  we  refer  to  Pre-Combat 
Readiness  and  Combat  Effectiveness . 

2 .  Indexing 

The  difficulty  in  defining  readiness  may  be  exceeded 
only  by  the  difficulty  of  measuring  it.   Many  attempts  have 
been  made  to  index  readiness,  but  to  date  none  have  been  suc- 
cessful or  achieved  wide  acceptance.   As  described  by  Barzily, 
Mar low,  and  Zacks,  in  their  "Survey  of  Approaches  to  Readi- 
ness," the  motivating  factor  common  to  all  attempts  is  to 
measure  the  effects  of  budgetary  changes  on  readiness.   They 
reviewed  a  number  of  readiness  indexes,  such  as  the  U.S. 
Navy  METRI  Project,  MARIS  Project,  MAXCAP  Models,  and  others 
and  found  that: 

The  readiness  indexes  posed  do  not  attain  the  desired 
objective.   They  are  usually  very  insensitive  to  changes 
that  occur  at  the  lower  echelons  and,  furthermore,  they 
are  generallv  improper  indexes  of  readiness.   [Ref.  11: 
p.  25] 

3 .  Economic  Question 

The  readiness  as  described  in  both  Definitions  and 
Indexing  sections  has  strong  economic  overtones.   The  approach 
in  this  study  is  to  treat  readiness  as  an  economic  question 
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(i.e.,  the  allocation  of  scarce  resources  of  training  time, 

material,  equipment,  logistics,  and  personnel  as  inputs  with 

the  desired  results  of  maximizing  outputs) .   The  true  output 

of  a  combat  organization  is  effectiveness  in  combat.   There 

is  obvious  difficulty  in  measuring  that  output.   Because  of 

this  difficulty  proxy  measures,  such  as  number  of  aircraft 

or  number  of  ships, are  generally  used.   The  difficulty  of 

measuring  output  was  recently  stated  in  a  study  by 

Rand  for  the  Assistant  Secretary  of  Defense/Manpower,  Reserve 

Affairs,  and  Logistics: 

The  output  of  an  Army  Maneuver/Firepower  Unit  is 
especially  difficult  because  such  units  have  no  single 
output  or  product  that  can  be  directly  related  to 
their  mission.   [Ref.  12:  p.  7] 

In  a  recent  study  by  Steven  L.  Funk,  several  observa- 
tions were 'made  which  have  application  here.   He  states  that 
although  peacetime  Army  readiness  is  an  honored  theme,  it  is 
not  always  practical.   This  is  a  result  of  preparedness  not 
being  the  actual  primary  peacetime  mission,  but  just  one  of 
several  competing  missions  that  take  resources  of  manpower, 
equipment,  and  training  time.   Funk  cites  major  factors  that 
detract  from  the  unit  being  able  to  train  for  combat  situations: 

(1)  Lack  of  resources  to  accomplish  requirements 
(i.e.,  skilled  personnel,  time,  ammunition, 
repair  parts,  and  facilities), 

(2)  changing  and  competing  priorities  that  fragment 
resources  and  negate  planning,  and 

(3)  a  climate  that  deprives  unit  leadership  of 
decision  "maneuver  space"  and  fragments  their 
attention  (i.e.,  centralized  decision  making/ 
control,  burdensome  administration,  low  unit 
cohesion  and  ambiguous  institutional  values) . 
[Ref.  13:  p.  6] 
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Although  the  previous  study "was  made  of  the  U.S. 
Army,  many  of  the  detractors  apply  equally  to  the  Marine 
Corps.   Some  policies  such  as  Unit  Rotation  (i.e.,  the  policy 
of  maintaining  unit  integrity  by  replacing  an  entire  battalion/ 
squadron  rather  than  replacing  individuals)  and  Deployment- 
Lock-On  periods  (i.e.,  the  policy  of  assigning  resources  to 
a  unit  well  in  advance  of  a  planned  deployment  and  then  main- 
taining and  replacing  those  resources  through  the  training 
cycle  and  into  deployment)  have  lessened  the  negative  effects. 
However,  in  the  Marine  Corps  the  problem  of  competing  priori- 
ties will  always  exist.   The  unit  commander  is  constrained  by 
many  elements  beyond  his  control  [Ref .  12J .   Thus  the  question 
of  unit  readiness  is  certainly  one  of  economic  priorities  or 
competing  priorities.   Readiness  to  a  large  degree  depends  on 
the  effective  resource  allocation  both  to  the  unit  and  within 
the  unit. 

C.   MCCRES  AS  THE  EVALUATION  SYSTEM 
1.   Research 

The  evaluation  system  employed  by  the  Marine  Corps 
has  undergone  much  study  from  the  standpoint  of  developing  an 
objective  versus  a  subjective  system  of  evaluation.   Much  of 
the  statistical  work  for  this  has  been  done  by  the  George 
Washington  School  of  Engineering  and  Applied  Science  [Refs. 
3,  4].   Additionally,  the  questions  of  evaluator  selection  bias 
were  analyzed  in  other  research  that  supported  this  study 
[Refs.  5,  6]. 
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As  a  result  of  the  George  Washington  University 
studies,  it  was  determined  MCCRES  had  the  following  merits: 

a.  Most  requirements  are  requests  for  descriptive  data 
and  not  for  judgments.   Judgments  were  previously 
made  by  defining  the  requirements  and  assigning 
weights . 

b.  The  details  of  the  doctrine  are  given  and  interpreted 
in  the  requirements,  thus  avoiding  the  possibility 

of  being  misinterpreted  or  forgotten  by  the 
evaluators . 

c.  The  execution  of  most  requirements  consumes  short 
time  periods  and  thus  the  evaluators '  memories  are 
not  overburdened. 

d.  Assigning  a  score  of  a  YES  or  NO  is  easier  than 
assigning  scores  on  any  other  scale. 

e.  The  set  of  the  requirements  exhausts  the  details 
of  the  executions.   [Ref.  3:  p.  25] 

Though  the  George  Washington  University  research  identifies 

merits  of  MCCRES,  it  does  not  address  the  question  of  what 

feedback  should  be  provided  to  the  evaluated  unit. 

2 .   Feedback 

The  purpose  of  this  research  was  to  determine  the 

types  of  feedback  that  are  most  useful  and  valuable  to  an 

evaluated  unit.   Feedback  is  a  necessary  part  of  any  control 

or  evaluation  system.   In  December,  1981,  B.  General  A.  A. 

Sordo,  then  Director  of  the  Marine  Corps  Training  Division, 

made  the  following  comment: 

The  evaluation  and  feedback  process  in  training  is 
one  where  we  need  to  do  substantial  work.   MCCRES 
helps  us  greatly.   In  the  future,  as  we  link  the 
individual  training  standards  of  our  ITS  System  to 
MCCRES  Mission  Performance  Standards,  we'll  be  able 
to  do  better!   But  above  and  beyond  that,  our 
measurements  of  adequacy  and  effectiveness  are  for 
the  most  part  decentralized  and  highly  subjective 
and  impressionistic.   In  the  long  run  we  rely  on 
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our  assessment  of  the  unit  or  organizational  capabili- 
ties as  the  best  gauge  of  adequacy  in  training 
preparation.   [Ref.  14:  p.  54] 


D.   RESEARCH  QUESTIONS 

As  a  result  of  the  need  to  define  specifically  the  feed- 
back and  information  needed  by  evaluated  units  as  the  result 
of  MCCRES ,  this  study  was  undertaken.   The  following  questions 
form  the  basis  for  this  study: 

1.  Primary 

a.  After  a  MCCRES,  what  information  should  be  fed 
back  to  the  evaluated  unit  commander? 

b.  In  what  form  should  the  evaluation  feedback  be? 

c.  What  channel (s)  should  be  used  to  provide  the 
feedback  information  to  the  unit  commander? 

2 .  Secondary 

In  addition  to  the  specific  questions  above,  there  are 
several  subsidiary  questions  which  arise  as  a  logical  conse- 
quence of  the  primary  questions.   They  involve  the  following 
topics:   (1)  validity  of  comparing  MCCRES  results  given  under 
differing  conditions,  (2)  MCCRES  contribution  to  effective 
training,  and  (3)  time  availability  and  followup  to  ensure 
correction  of  deficiencies  identified  during  the  MCCRES. 

To  properly  answer  the  questions  posed,  the  authors 
took  a  multi-disciplinary  approach.   The  management  control 
and  evaluation  literature  can  provide  a  great  deal  of  insight 
toward  understanding  the  complexities  of  the  information 
needs.   Thus  the  literature  of  the  two  fields  is  exDlored  in 
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detail  in  the  next  chapter.   In  addition  much  work  has  been 
done  in  the  area  of  modeling  effectiveness.   This  has  been 
done  from  the  perspective  of  applied,  conceptual,  statistical 
and  economic  models.   Since  these  provide  rich  theories  for 
the  understanding  of  the  measurement  and  indexing  of  effec- 
tiveness, these  concepts  are  explored. 

E.   SUMMARY 

This  chapter  was  designed  to  provide  a  history  of  MCCRES , 
explain  the  need  for  such  a  system  from  the  Marine  Corps 
perspective,  give  a  basic  description  of  the  system,  and 
provide  a  detailed  understanding  of  the  inherent  difficulties 
of  efforts  to  evaluate  the  concept  of  readiness. 

Readiness  is  approached  from  the  standpoint  of  economics — 
the  application  of  scarce  resources  (inputs)  of  time,  men, 
materials  to  the  process  of  simulated  combat.   The  unit  com- 
mander controls  the  organization  with  a  highly  sophisticated 
management  control  system  in  accordance  with  the  external  and 
internal  objectives  and  policies.   MCCRES  is  a  unit  evaluation 
system  which  evaluates  the  management  control  process  of  that 
organization.   The  primary  questions  are  then  presented  as 
the  basis  of  this  research.   The  following  chapters  briefly 
explore  the  management  control  systems  and  evaluation 
literature  with  specific  emphasis  on  feedback  mechanisms. 
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III.   MANAGEMENT,  MANAGEMENT  CONTROL,  EVALUATION, 

AND  FEEDBACK 


The  topics  to  be  covered  in  this  chapter  are  as  follows: 
first  the  topic  of  management  is  described  in  a  general 
context;  management  control  and  its  implications  for  feedback 
are  explored;  then,  a  series  of  selected  evaluation  theories 
are  presented  in  detail.   Finally  a  series  of  evaluation 
models  with  application  to  the  military  are  provided.   Empha- 
sis in  these  models  is  on  feedback.   A  summary  of  feedback 
as  developed  in  the  models  is  presented. 

A.   MANAGEMENT 

Management  is  described  as:   (1)  "art  or  act  of  managing; 
conduct;  control;  direction.   (2)   Judicious  use  of  means  to 
accomplish  an  end;  skillful  treatment"  [Ref.  15].   Webber 
describes  it  in  many  contexts—experience ,  training,  practice, 
theory,  art,  and  science  [Ref.  16].   The  simplest  definition 
and  one  often  heard  is  the  art  of  getting  things  done  through 
people.   The  elements,  however,  that  all  descriptions  of 
management  have  in  common  are  that  managers  engage  in  two 
important  activities:   planning  and  control  [Ref.  17].   It 
is  important  to  make  the  distinction  clearly  between  the  terms 
management  and  leadership.   The  military  has  long  recognized 
that  the  term  "leader"  has  much  broader  implications  than 
that  of  manager.   The  unit  is  more  than  the  sum  of  its  parts; 
and,  therefore,  the  military  commander's  responsibility  is 
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not  only  for  the  technical  and  tactical  proficiency  of  the 
unit,  but  for  the  overall  cohesive  functioning  of  the  organi- 
zation [Ref .  18] .   In  this  context  it  is  appropriate  to  say 
that  management  skills  are  a  necessary  subset  of  leadership 
and  that  the  good  leader  is  necessarily  a  good  manager,  how- 
ever, the  converse  of  that  statement  is  not  necessarily  true. 

B.   MANAGEMENT  CONTROL 

Robert  Anthony  placed  management  control  in  a  hierarchi- 
cal framework  with  Strategic  Planning  above  and  Operational 
Control  below  Management  Control.   He  described  this  classi- 
fication for  any  large  organization  where  these  functions  are 
present. 

Strategic  planning  is  the  highest  level  of  management 

where  the  long  range  objectives,  policies,  and  plans  of  the 

organization  are  made. 

Strategic  planning  is  the  process  of  deciding  on 
objectives  of  the  organization,  on  changes  in  the 
objectives,  on  resources  used  to  attain  these 
objectives,  and  on  the  policies  that  are  to  govern 
the  acquisition,  use,  and  disposition  of  these 
resources.   [Ref.  19:  p.  16] 

The  management  control  level  involves  middle  management, 

and  the  timeframe  is  shorter,  about  12  months. 

Management  control  is  the  process  by  which  managers 
assure  that  resources  are  obtained  and  used  effec- 
tively and  efficiently  in  the  accomplishment  of  the 
organization's  objectives.   [Ref.  19:  p.  17] 

The  lowest  level  of  control  is  operational  where  the  day- 
to-day  functioning  of  the  organization  is  accomplished.   The 
implementation  of  plans  and  policies  are  executed  at  this  level 


18 


Lebas  describes  management  control  as  an  enlarged  feedback 
loop  which  affects  human  behavior  and  performance  [Ref.  20]. 
Hofstede  also  provides  a  useful  definition  of  management  con- 
trol "...as  a  pragmatic  concern  for  results,  obtained 
through  people"  [Ref.  21:  p.  193]. 

Given  the  purpose  of  this  study  is  to  identify  feedback 

needed  for  improved  resource  allocation,  it  is  important  to 

focus  on  the  terms  "effectiveness"  and  "efficiency",  which 

are  present  in  many  of  the  definitions  of  management  control. 

In  this  context  the  terms  effectiveness  and  efficiency  are 

defined  as  follows: 

Effectiveness .. .relates  to  the  accomplishment  of  the 
cooperative  purpose. . .when  a  specific  desired  end  is 
attained  we  shall  say  that  the  action  is  effective. 
[Ref.  19:  p.  27] 

Ef f iciency . . .ref ers  to  the  engineering  sense  of... 
the  optimum  relationship  between  given  inputs  and 
outputs.   [Ref.  19:  p.  27] 

Lebas  identifies  the  three  major  schools  of  thought 
on  management  control:   organizational  process  approach, 
information  economics  approach  and  behavioral  approach. 

1.   Organizational  Process  Approach 

The  Organizational  Process  Approach  is  that  taken 
by  Anthony.   He  divides  the  feedback  loop  into  three  distinct 
parts,  described  previously  as  strategic  planning  and  manage- 
ment control  and  operational  control.   This  approach  reduces 
the  human  motivational  problem  to  what  Anthony  calls  "gcal 
congruence"  or  the  various  ways  in  which  the  manager  can  be 
encouraged  to  take  actions  which  are  in  the  best  interest  of 
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the  company.   Achieving  effective  and  efficient  use  of  re- 
sources is  accomplished  through  procedures  within  the 
organization  such  as  performance  evaluation,  reporting,  and 
budgeting  [Ref .  20] . 

2 .  Information  Economics  Approach 

The  Information  Economics  Approach  is  more  abstract. 
It  proposes  that  only  models  based  on  expected  utility  maxi- 
mization need  to  be  considered.   A  set  of  subjective  probabili- 
ties are  assumed  to  be  known  by  the  decision-maker  and  that 
the  decision-maker  will  always  act  to  maximize  utility.   In 
this  approach,  good  performance  is  described  as  the  action 
which  minimizes  the  end  result  differences  between  the  expected 
outcomes  and  the  actual  outcomes.   The  emphasis,  however,  is 
on  the  decision  model  not  the  outcome.   A  further  aspect  of 
the  theory  is  its  emphasis  on  maximizing  the  expected  utility. 
This  leads  to  the  conclusion  that  the  control  process  is  a 
prior  phenomenon  that  suggests  management  spend  more  time 
preparing  decisions  than  evaluating  outcomes  [Ref.  20]. 

The  major  limitation  to  the  Information  Economics 
Approach  is  that  it  is  all  based  on  the  Savage-Rational  Man 
Model  and  that  an  effort  will  be  made  to  maximize  utility  in 
every  case.   Although  an  interesting  theoretical  model,  it 
relies  heavily  on  the  calculus  of  maximizing  utility  in  each 
and  every  case  [Ref.  20] . 

3 .  Behavioral  Approach 

This  is  actually  a  loosely  structured  set  of  ap- 
proaches to  the  concept  of  management  control  taking  ideas 
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from  many  theories.   Two  descriptive  ideas  that  it  entails 

from  Herbert  Simon  are: 

Satisf icing ...  interrupting  the  action  selection  process 
as  soon  as  the  first  acceptable  action  has  been  found. 

And 

Bounded  Reality. . .the  notion  that  man  is  a  piecewise 
rational.   [Ref.  20] 

Other  ideas  presented  by  Ouchi  and  referred  to  by 

Lebas  have  shown  that: 

...rigorous  output  control  is  the  most  effective  way 
to  induce  good  performance  in  managers.   [Ref.  20] 

Lebas  views  the  real  value  of  behavioral  approaches 
in  their  diversity  and  being  based  on  observations  of  people 
in  real  situations.   The  limitations  come  from  their  multi- 
plicity and  that  creates  difficulty  in  setting  any  sort  of 
coherent  guidelines  for  a  manager  to  employ  [Ref.  20]. 
4 .   Integrated  Approach 

The  previous  broad  structures  of  management  control 
system  have  been  integrated  by  Ansari  in  what  he  refers  to 
as  an  operational  systems  concept  [Ref.  22].   He  views  the 
previous  perspectives  on  management  control  systems  in  two 
versus  three  broad  categories.   Grouping  the  structural  view 
(Operational  Process  and  Information  Economics)  and  the 
behavioral  views : 

a.   Two  Perspectives 

Ansari  describes  the  structural  view  of  management 
control  as  that  adopted  by  researchers  in  cybernetics,  account- 
ing, and  management  information  systems — those  that  concentrate 
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primarily  on  the  information  and  communications  aspects  of 
control  systems.   The  behavioral  approaches  are  identified 
as  those  that  are  based  primarily  on  human  behavior  in  organi- 
zations and  regard  control  as  a  problem  of  encouraging  sub- 
ordinates to  achieve  performance  goals.   In  this  context  he 
uses  control  systems  to  describe: 

...those  arrangements  and  actions  designated  to 
facilitate  its  members  to  achieve  higher  performance 
with  least  unintended  consequences.   [Ref.  22:  p.  102] 

b.  Elements  of  Management  Control  Systems 

There  is  agreement  among  most  authors  reviewed 
from  both  the  structural  and  behavioral  schools  that  all 
management  control  systems  consist  of  two  elements.   The 
first  being  "an  information  network  which  prescribes  the 
rules  for  measurement,  collection,  processing,  and  transmission 
of  information"  [Ref.  22:  p.  102].   This  element  causes  the 
information  on  performance,  goals,  outputs,  and  exceptions 
from  plans  to  be  transmitted  to  managers.   The  second  ele- 
ment is  the  set  of  social  relationships  through  which  the 
control  system  achieves  the  organizational  goals  [Ref.  22J . 

c.  Joint  Consideration 

Ansari  argues  that  management  control  is  actually 
best  approached  from  a  combination  of  the  structural  and  be- 
havioral ideas.   He  declares  that  the  current  phase  of  design 
is  more  situational  and  that  it  focuses  on  the  more  important 
issue  of  improving  performance  instead  of  the  narrow  con- 
cept of  constraining  behavior.   The  information  that  is  pro- 
vided passes  through  two  phases.   First,  it  must  be  perceived 
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before  it  can  be  used  and  second  not  only  managers,  but 
subordinates,  must  also  use  information  for  self-evaluation 
and  guidance  of  performance.   He  further  identifies  five  con- 
siderations which  are  characteristic  of  all  information 
structures. 

1.  Nature  of  Measures — the  subordinates'  motivation 
is  affected  by  the  completeness,  objectivity, 
and  influence  of  the  characteristics  of  perform- 
ance measures. 

2.  Source  and  Order  of  Presentation—The  source  of 
the  information  must  be  credible  and  the  order 
of  presentation  can  change  the  perceptions  of 
the  information. 

3.  Timing — Both  the  speed  and  frequency  are  impor- 
tant, too  long  an  interval  may  cause  the  user 
to  lose  interest  or  be  distracted. 

4.  Route--The  route  may  alter  the  information  and 
thus  the  sender  and  receiver  may  not  be  sharing 
the  same  information. 

5.  Shared  Information — The  others  with  whom  the 
information  is  shared  thus  may  affect  the 
recipient's  view  of  the  information  with  regard 
to  fairness  and  accuracy.   [Ref.  22] 

All  of  these  considerations  must  be  made  by  the 
designer  in  any  information,  evaluation  system.   To  summarize 
the  characteristics  of  the  integrated  model,  this  takes  into 
account  both  the  structure  of  the  organization  and  its  infor- 
mational systems  as  well  as  the  social  side  considering 
subordinate  personality  and  leadership  style.   The  designer 
of  a  management  control  system  should  combine  components 
(such  as  an  evaluation  system)  in  such  a  way  as  to  minimize 
cognitive  conflicts  and  encourage  behavior  which  resolves 
conflict  with  positive  results  for  the  organization.   It 
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should  also  be  kept  in  mind  that  if  rewards  for  performance 
are  too  highly  contingent  on  outputs  of  the  information  struc- 
ture, the  greater  the  chance  that  there  will  be  controversy 
over  the  output  [Ref .  22] .   A  good  example  of  this  contro- 
versy is  discussed  by  Anthony  Hopwood  in  the  way  in  which 
accounting  data  was  used  in  the  performance  evaluation  of 
managers . 

Based  upon  a  study  he  conducted,  Hopwood  identi- 
fied four  ways  in  which  accounting  data  was  used  in  evaluation. 

(1)  Budget  Constrained  Style — required  to  meet  a  budget 
but  not  concerned  about  costs. 

(2)  Budget  Profit  Style — concerned  with  both  meeting  a 
budget  and  with  costs. 

(3)  Profit  Conscious  Style--concerned  with  costs  but  not 
with  meeting  a  budget. 

(4)  Non-Accounting  Style — not  concerned  with  meeting  a 
budget  or  with  costs.   [Ref.  23] 

He  concluded  that  based  on  management  style,  many  dysfunc- 
tional behaviors  may  occur  as  the  result  of  emphasis  on 
accounting  data  results  in  evaluations  [Ref.  23]. 
5 .   Other  Thoughts  on  Management  Control 

Management  control  contains  many  rich  conepts  which 
play  a  great  part  in  the  theory  of  evaluations .   San  Miguel 
has  made  several  interesting  observations  on  management 
control: 

The  end  is  a  system  that  enables  managers  to  make  sound 
decisions  as  to  the  efficient  and  effective  allocation 
of  human,  physical,  and  financial  resources  to  attain 
the  objectives  of  the  organization.   [Ref.  24:  p.  177] 
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San  Miguel  further  characterizes  management  science's 
attempts  to  deal  with  modeling  of  complex  control  systems. 
He  characterizes  the  quantitative  and  economic  decision  tools' 
attempt  to  simplify  the  decision  making  in  large  scale  organi- 
zations as  falling  short  of  their  objectives — primarily  be- 
cause of  the  lack  of  knowledge  of  human  behavior  and  the 
ability  to  quantify  it.   He  further  cites  measurement  and 
communications  to  be  at  the  center  of  control  systems,  internal 
planning,  and  reporting.   There  is  general  agreement  in  the 
management  control  literature  that  measurement  and  communica- 
tions systems  have  an  important  impact  on  the  behavior  of 
individuals  in  organizations,  their  motivations  and,  thus, 
their  performance.   Therefore,  the  areas  of  measurement  and 
evaluation  are  legitimate  concerns  for  the  design  of  any 
management  control  system  [Ref.  24] . 
6 .   Summary 

The  intent  in  this  section  was  to  set  the  stage 
through  the  discussion  of  the  broader  topics,  management  and 
mangement  control,  for  what  is  to  be  developed  in  the  next 
section  on  evaluation.   Management  control  is  a  complex  topic 
which  encompasses  the  broad  aspects  of  all  the  control  sys- 
tems imposed  on  the  organization  through  strategic  plans  and 
policies  as  well  as  those  developed  within  to  assist  managers 
in  internal  control. 

C.   EVALUATION  OVERVIEW 

Evaluation  models  directed  at  readiness  all  embody  a 
common  goal.   Each  in  its  own  particular  way  attempts  to 
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identify  a  definitive  measure  or  some  particular  relative 
measure  of  the  unit's  Pre-D-Day  preparedness  for  combat. 
Preparedness  may  be  viewed  as  readiness  which  acts  as  a  proxy 
measure  for  the  military  unit's  ability  to  perform  in  combat. 
Similar  efforts  are  made  to  evaluate  the  ability  to  perform 
in  civilian  organizations.   In  its  simplest  form,  output  is 
measured  by  production  level  or  quantity  produced  by  a  produc- 
tion facility  in  a  given  time  period.   Various  attempts  are 
also  made  to  measure  profitability  of  the  firm.   Microeconomic 
theory  assumes  that  each  firm  is  attempting  to  maximize  profit 
[Ref.  25].   Therefore,  some  measure(s)  are  applied  to  the 
fiscal  results  of  each  period  to  determine  the  financial 
performance  of  the  organization  during  the  period  [Ref.  25]. 
Similarly  it  can  be  assumed  that  each  unit  tries  to  maximize 
its  readiness.   This  section  reviews  efforts  that  have  been 
made  to  evaluate,  measure,  and  model  performance  both  in 
theory  and  practice. 

1.   Evaluation  Defined 

The  term  evaluation  is  defined  differently  by  various 
authors;  however,  Stuff lebeam  et  al.  brings  together  three 
generally  accepted  definitions  of  evaluation  and  provides  a 
comparison . 

a.   Measurement  Definition 

Evaluation  is  identical  to  measurement.   It 
builds  on  attempts  to  measure  psychological  attributes  or 
characteristics  [Ref.  26:  pp.  9-10]. 
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b.  Congruence  Definition 

Evaluation  is  a  comparison  of  the  congruence 
between  performance  and  objectives.   This  provides  emphasis 
on  objective  performance  also  [Ref .  26] .   The  measurement 
definition  considered  evaluation  to  occur  after  the  fact  and 
to  measure  some  attribute.   In  contrast,  the  congruence 
definition  refers  to  on-going  evaluation  in  meeting  the 
objectives.   According  to  Stufflebeam,  organizations  pick 
objectives  which  are  specific  and  have  objective  measurable 
results.   To  make  the  congruence  definition  work,  there  must 
be  some  selection  of  objectives  to  be  measured.   Stufflebeam 
further  indicates  that  the  congruence  definition  creates  a 
need  for  the  evaluator  to  find  short  term  measures  of  per- 
formance.  These  indicators,  identifiable  outputs,  or  behaviors 
come  to  be  viewed  as  performance.   These  identifiable  outputs 
become  the  ultimate  criteria  of  all  organizational  decisions. 
The  congruence  definition  has  advantages,  but  the  need  to 
find  measurable  objectives  creates  problems  [Ref.  26]. 

c.  Judgment  Definition 

The  professional  judgment  definition  allows  full 
evaluation  of  all  organizational  attributes,  both  quantifiable 
and  non-quantifiable,  and  is  easy  to  implement  [Ref.  26]. 
This  definition  is  often  applied  where  the  dimensions  to  be 
measured  are  difficult  to  define  and  more  difficult  to  quan- 
tify.  The  judgment  definition  has  its  problems.   The  evalua- 
tion is,  by  definition,  based  upon  the  judgment  of  an  individual 
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which  lacks  objectivity  identifying  what  data  are  used  and 
how  they  are  used  [Ref.  26]. 

Table  3.1,  taken  from  Stuff lebeam,  provides  a 
comparison  of  advantages  and  disadvantages  of  each  definition. 

2 .  Principles  of  Evaluation 
For  an  evaluation  to  be  effective  it  must  be  consis- 
tent with  the  purposes,  objectives,  and  goals  of  the  activity 
being  evaluated  [Refs.  5,27,28]: 

a.  Evaluations  must  be  conducted  in  terms  of  purpose — 
the  evaluator  and  the  evaluated  must  fully  participate 
and  work  for  the  common  goal . 

b.  Evaluation  must  be  cooperative — all  involved  as  both 
the  evaluators  and  evaluated  must  fully  participate 
and  work  for  the  common  goal. 

c.  Evaluation  must  be  continuous  —  it  must  be  on-going; 
a  one  time  effort  with  no  followup  is  an  affront  to 
the  professional  concepts. 

d.  Evaluation  must  be  specific — specificity  is  the  key, 
generalizations  do  little  good  to  help  remedy  defi- 
ciencies or  to  identify  true  strengths. 

e.  Evaluation  must  provide  the  means  and  focus  for 
trainers  to  be  able  to  appraise  themselves,  their 
practices,  and  their  products. 

f .  Evaluation  must  be  based  upon  uniform  and  objective 
methods  and  standards . 

3 .  Summary 
Definitions  and  principles  of  evaluation  were  pre- 
sented in  this  section.   A  single  commonly  agreed  upon  defini- 
tion of  evaluation  does  not  exist.   There  is  more  general 
agreement  on  the  principles  of  evaluation  and  to  a  large 
degree  these  embody  a  common  sense  approach  for  the  usefulness 
of  any  evaluation.   Next,  four  evaluation  theories  are  presenter. 
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TABLE  3.1 

Advantages  and  Disadvantages  Accruing  from  Different 
Definitions  of  Evaluation 


Advantages 

Builds  directly  on  scientific 
measurement 

Objective 

Reliable 

Data  mathematically 
manipulatable 

Norms  and  standards 
emerge 

High  degree  of  integration  with 
the  process 

Data  available  on  process  and 
structure 

Possibility  of  feedback 


Easy  to  implement 

Brings  all  variables 
into  consideration 

Takes  experience  and  expertise 
into  account 

No  time  lag  while  waiting  for  data 
analysis 


Disadvantages 

Narrow  instrumental  focus 

Inflexible  because  of  time 
and  cost  to  produce  new 
instruments 

Obscures  judgments  and  the 
criteria  for  maning  them 

Eliminates  variables  currently 
considered  as  not  measurable 
or  labels  them  unimportant 

Places  evaluator  In  tecnnical 
role 

Focuses  narrowly  on  objectives 

Elevates  behavior  as  the  ultimate 
criterion  of  every  action 

Focuses  on  evaluation  as  a 
terminal  process 

Dictated  mainly  because  of 
ignorance  or  lack  of  sophistication 

Questionable  reliability 

Questionable  objectivity 

Not  susceptible  to  ordinary 
scientific,  prudential  measures 

Both  data  and  criteria  are 
ambiguous 

Generalization  very  difficult 


urce:     Adopted  from  Ref.  26:  p.  15. 
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D.   EVALUATION  MODELS 

There  are  diverse  models  and  applications  of  evaluation. 
This  discussion  is  not  intended  to  be  all  inclusive.   What 
is  intended  is  to  introduce  a  variety  of  models  which  exist 
and  to  show  their  relationship  to  the  questions  pursued  in 
this  research. 

1.   Management  by  Objective  (MBQ) 

In  recent  years  probably  no  system  of  personnel 

management  has  gained  more  attention  than  Management  by 

Objective  introduced  by  Peter  Drucker  [Ref.  29].   The  MBO 

approach  is  applied  to  individual  as  well  as  organizational 

evaluation  systems. 

The  MBO  Approach  assumes  that  one  would  perform  more 
effectively  because  you  have  planned  your  own  objec- 
tives and  could  control  your  own  behavior.   [Ref.  16: 
p.  316] 

MBO  takes  a  humanistic  or  human  values  approach  to 

managing  people.   The  basic  steps  in  MBO  are  as  follows: 

1.  The  subordinate  proposes  goals  for  the  next  time 
period. 

2.  The  subordinate  and  superior  discuss,  modify,  and 
reach  agreement  on  the  goals . 

3.  Periodic  formal  and  frequent  informal  review  of  the 
goals  and  progress  toward  the  goals. 

4.  Subordinate  reports  on  performance  at  the  end  of 
the  period. 

5.  Repeat  the  cycle.   [Ref.  16:  p.  316] 

MBO  deals  well  with  a  number  of  problem  areas  in  evaluation 
systems.   It  is  designed  to  facilitate  two-way  communication. 
MBO  has  found  success  in  both  the  private  and  public  sector. 
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In  their  article  on  "Employee  Growth  Through  Performance 
Management,"  Beer  and  Ruhr  found  success  with  employment  of 
MBO  at  Corning  Glass  Co.  [Ref.  30] .   In  addition,  the  Depart- 
ment of  Health,  Education,  and  Welfare  found  it  a  most  valua- 
ble system  in  pursuing  management  and  control  of  their  many 
diversified  programs  [Ref.  31].   Figure  3.1  provides  a  simple 
model  of  how  MBO  is  designed.   It  should  be  noted  that  the 
MBO  model  focuses  on  performance  as  identified  and  measured 
through  objectives  agreed  upon  by  supervisor  and  subordinate 
[Ref.  32]. 

Feedback  in  the  MBO  System  is  scheduled  at  regular 
intervals.   The  meetings  can  be  used  to  compare  results  to 
the  objectives  set  as  well  as  to  help  make  future  objectives 
more  realistic.   The  objectives  themselves  are  determined  by 
higher  level  management  in  coordination  with  the  employee 
or  unit.   To  a  great  extent  the  comparison  of  goals  set  and 
achieved  are  reported  by  the  individual  being  evaluated. 
Typically  the  manager  submits  an  evaluation  of  each  objective 
based  on  the  following  classification  system: 

1.  Satisfactory 

2.  Minor  Problem 

3.  Major  Problem 

Feedback  is  handled  by  face  to  face  conferences  between  the 
evaluator  and  employee. 

In  a  recent  Marine  Corps  Gazette  article,  Major  James 
M.  Clarke  reflected  his  good  experiences  with  the  MBO  merit 
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Figure    3.1.    Sample  MBO  Model 
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pay  system  used  with  high  level  GS  employees.   He  referred 

to  MBO  in  the  context  of  a  personnel  evaluation  system; 

however,  it  also  has  application  here: 

To  improve  performance  counseling,  we  must  get  away 
from  subjective  judgments  through  a  more  formal  system 
which  concentrates  on  objectives.   The  merit  pay 
system  provides  a  good  model.   [Ref.  33:  p.  47] 

Major  Clarke's  article  provided  support  for  the  MBO 
method  because  clear  objectives  are  established  and  regular 
communications  are  encouraged.   In  the  Marine  Corps'  personnel 
system,  as  in  other  areas  of  inspection  and  evaluation,  pro- 
fessional judgment  has  historically  been  used  as  the  primary 
method  of  evaluation  and  lack  of  specific  objectives  has 
reinforced  the  reliance  on  the  judgmental  method. 

MBO  places  a  great  emphasis  on  the  objectives  and  the 
regular  feedback  on  how  the  evaluated  worker  is  measuring  up 
to  those  objectives  [Ref.  16].   MBO  further  emphasizes  dis- 
playing the  individual's  strengths  and  weaknesses  to  himself, 
rather  than  to  others.   Although  MBO  emphasizes  timely, 
accurate,  and  objective  feedback,  the  details  of  that  feed- 
back can  only  be  spelled  out  when  a  system  is  input  to  a 
specific  organization  [Ref.  30].   It  further  encourages  the 
feedback  be  given  on  a  continual  basis,  not  "saved  up  for  an 
end  of  the  year  inquisition"  [Ref.  16:  p.  317]. 
2 .   Evaluation  and  Authority 

A  comprehensive  and  ongoing  study  of  evaluation  and 
authority  was  conducted  by  Scott  and  Dornbusch  (19  77)  .   In 
their  study  of  professional  organizations,  results  were 
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gathered  from  university  faculty  members,  clergy,  public 

school  teachers,  principals,  nurses,  and  others.   They  found: 

Organizations  are  power  structures  in  which  some  partici- 
pants give  differential  access  to  organizational 
rewards  and  penalties  in  order  to  control  other 
participants.   [Ref.  34:  p.  134] 

The  Scott-Dornbusch  studies  present  a  conceptual  model 
of  the  relationship  of  evaluation  and  authority.   As  a  result  o 
these  studies,  they  developed  the  model  presented  in  Figure 
3.2.   The  model  is  a  systems  analysis  approach  which  provides 
for  monitoring  and  regulating  by  means  of  a  feedback  loop. 
This  approach  concentrates  on  the  involvement  of  each  actor 
in  the  process.   The  actors  in  the  Scott-Dornbusch  model  are 
described  as  follows:   The  allocator  gives  the  responsibility 
for  performing  the  task.   The  criteria  setter  is  concerned 
with  assessing  the  effectiveness  and  efficiency  of  a  task 
performance.   To  a  large  measure  the  criteria  is  determined 
by  the  goal  toward  which  the  performance  is  directed.   The 
determination  of  which  information  to  use  in  order  to  arrive 
at  a  performance  evaluation  is  the  responsibility  of  the 
sampler .   The  appraiser  uses  the  information  on  the  sampled 
indicators  and  transforms  the  observed  values  into  scores. 
The  goal  of  the  evaluation  process  is  to  provide  a  measure 
of  the  performer  based  on  performance  and  outcome  [Ref.  34 J . 
The  manager  must  get  each  of  these  individuals  in  the  decision- 
making process  and  coordinate  their  activities.   Emphasis  is 
placed  in  this  model  on  communications  among  the  right-holders. 
The  right-holders  are  those  given  authorized  power  or  authority 
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Figure    3.2        Model  of  Evaluation  Process  with  Communication 
Links  Among  Right- Holders. 
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rights  by  the  organization.   The  authority  rights  are  granted 
to  the  allocator,  criteria  setter,  sampler  and  appraiser. 
There  are  a  variety  of  interconnecting  links  which  may  exist 
between  these  rights-holders  which  define  the  authority 
structure  within  any  given  organization.   Thus  the  formal 
evaluation  process  is  based  on  the  structure  of  the  inter- 
connection among  the  right-holders  [Ref.  34]. 

The  Evaluation  and  Authority  Feedback  Model  describes 
performance  by  direct  and  indirect  measures.   An  example  of 
a  direct  measure  would  be  the  number  of  units  produced  as  a 
measure  of  labor  productivity.   An  indirect  measure  is  the 
number  of  professional  papers  written  to  evaluate  the  produc- 
tivity of  a  scientist  or  scholar  [Ref.  34].   The  variety  of 
samples  requires  judgment  on  the  part  of  the  evaluator  to 
determine  what  is  high  or  low  performance  so  judgment  must 
be  used  by  the  evaluator.   Thus,  accurate  appraisal  of  the 
sample  taken  requires  a  complete  knowledge  of  the  task,  the 
performance,  and  the  specific  circumstances.   Based  on  all  of 
these  measures  it  is  decided  what,  if  anything,  should  be 
conveyed  to  the  performer  concerning  quality  of  the  task 
performed  [Ref.  34]. 

3 .   Practice  of  Program  Evaluation 

Anderson  and  Ball  take  an  applied  approach  to  program 
evaluation.   They  identify  six  specific  purposes  of  program 
evaluation  [Ref.  35]: 

1.  to  contribute  to  decision  about  program  installation, 

2.  to  contribute  to  decisions  about  program  continuation, 
expansion,  or  certification, 
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3.  to  contribute  to  decisions  about  program  modifications, 

4.  to  obtain  evidence  to  rally  support  for  a  program, 

5.  to  obtain  evidence  to  rally  opposition  to  a  program, 

6.  to  contribute  to  the  understanding  of  basic  psychologi- 
cal, social,  and  other  programs.   [Ref .  35:  p.  4] 

Anderson  and  Ball  emphasize  the  practice  aspect  that 
evaluations  can  address  a  wide  variety  of  questions  and  pro- 
vide many  useful  services.   Their  experience  is  primarily 
with  social  programs,  but  it  is  applicable  to  this  study. 
They  identify  a  total  of  seven  types  of  evaluation  methods: 

1.  The  experimental  and  quasi-experimental  study, 

2.  Correlation  methods, 

3.  Surveys, 

4.  Personnel  or  client  assessment, 

5.  Systematic  expert  judgment, 

6.  Clinical  or  case  studies, 

7.  Informal  observation  and  testimony. 

The  real  richness  of  the  ideas  presented  by  Anderson 
and  Ball  involve  their  emphasis  on  feedback  which  they  refer 
to  as  the:   Communication  and  Dissemination  of  Results.   They 
emphasize  the  need  for  communications  of  the  results  to  be 
bi-directional.   The  effective  communications  of  results  are 
viewed  as  a  sign  of  the  positive  health  of  the  program.   Evalu- 
ators  should  take  pride  in  presenting  and  disseminating  their 
results.   The  dissemination  is  more  than  merely  a  phase  tacked 
on  the  end  of  a  program;  it  involves  more  than  simply  telling 
the  findings.   At  a  minimum,  what  the  dissemination  involves 
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is  that  the  results  should  include  information  about  the 
evaluation  plans  and  procedures  as  well  as  the  findings. 
There  are  also  several  criteria  provided  to  guide  the  dis- 
semination to  various  audiences.   First,  if  an  evaluation  is 
worth  doing,  other  groups  than  the  evaluated  organization  have 
some  interest  in  finding  out  about  the  results.   Second,  is 
that,  given  different  audiences,  several  mediums/methods  of 
dissemination  may  be  called  for.   Anderson  and  Ball  classify 
some  fifteen  different  audiences  that  should  at  least  be  con- 
sidered for  dissemination  of  results  [Ref .  35] . 

Use  of  results  is  another  consideration  once  the 
evaluation  is  completed.   The  logical  expectation  is  that  the 
decision  maker  will  use  the  results  to  make  rational  future 
decisions.   Weiss  lists  several  reasons  why  that  is  not  always 
the  case.   The  nature  of  the  organization  is  sometimes  opposed 
to  the  use  of  the  results;  however,  if  the  decision  maker  and 
the  evaluator  have  maintained  close  communications  throughout 
the  process,  the  potential  objections  can  be  reduced  [Refs. 
35,36].   Particular  consideration  must  be  given  not  only  to 
factual  evaluation  of  each  program,  but  also  the  salesmanship 
aspect  of  presenting  the  evaluations. 

Anderson  and  Ball  have  adapted,  from  a  paper  by  J.  S. 
Berke  a  structure  which  they  suggest  should  be  applied  when 
writing  evaluation  reports. 

•Brevity  and  clarity.   Critical  findings  should  be 
summarized  clearly  and  simply  at  the  outset. 

•Timeliness — to  be  useful  and  utilized,  results  must 
be  reported  according  to  other  peoples '  schedules  and 
not  the  evaluator' s  research  clock. 
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•Interim  products  and  reports—these  help  prepare  deci- 
sion makers  for  the  impact  of  larger,  later  evaluation 
reports.  Besides,  they  can  allow  preliminary  planning 
for  use  even  before  the  final  report  is  available. 

•Responsiveness--he  notes  here  that  traditional  researchers 
tend  to  make  questions  more  interesting,  designs  more 
elegant,  the  analysis  more  comprehensive,  and  the  use  of 
recommendations  more  guarded  than  necessary. 
[Ref.  35:  pp.  107-108] 

In  summary,  the  program  evaluator  has  the  responsibility 
to  push  the  results  of  the  evaluation  and  to  bring  them  to 
the  attention  of  all  those  who  should  be  made  aware  [Ref.  35]. 
4 .   Critical  Incident  Evaluation 

Critical  Incident  Evaluation  Program  involves  manage- 
ment writing  regular  reports  on  the  performance  they  observe. 
The  critical  incident  process  has  as  its  roots  the  idea  that 
actual  behavior  should  be  appraised,  not  traits  of  behavior. 
It  is  an  attempt  to  justify  ratings  based  on  specific  inci- 
dents that  provide  support  evidence  for  the  evaluation. 
Levinson  places  strong  emphasis  on  the  need  to  evaluate  on 
how  things  get  done  and  not  just  results  [Ref.  37] .   Levinson 
argues  that  in  reality  people  are  evaluated  on  the  "how",  but 
many  are  led  to  believe  that  they  are  just  judged  based  on 
results . 

Thus,  the  profit  oriented  manager  pulls  his  company 
out  of  the  "red",  but  is  criticized  for  the  methods  used  to 
do  it.   Levinson  cites  many  deficiencies  with  current  evalua- 
tion systems.   One  is  that  no  matter  how  well-defined  the 
dimensions  for  appraising  performance  on  a  quantitative  basis, 
judgments  on  performance  are  always  subjective  and  impression- 
istic [Ref.  37].   Second,  delay  in  giving  feedback  creates 
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frustration  both  when  performance  is  good  and  credit  is  de- 
served, and  when  performance  is  bad  and  criticism  is  rendered 
long  after  the  performance.   Levinson  views  performance 
evaluation  not  as  a  technique,  but  a  process  involving  both 
people  and  data.   Thus,  the  process  is  inadequate  [Ref.  37]. 

Levinson  proposes  that  an  effective  critical  incident 
model  be  composed  of  the  following  [Ref.  37J  : 

1.  A  dynamic  job  description — one  which  amplifies 
statements  of  job  responsibility  and  desired 
outcomes  by  describing  the  emotional  and 
behavioral  topography  of  the  task. 

2.  A  critical  incident  process — this  requires  jointly 
setting  objectives  and  discussing  each  piece  of 
behavior  that  is  judged  good  or  bad  by  the  manager. 

3.  A  psychological  support  system — to  accomplish 
this,  the  manager  must  learn  to  cope  with  feelings 
or  guilt  over  the  appraisal.   He  calls  this  upward 
appraisal  concept  in  which  managers  who  develop 
employees  through  their  effective  appraisal  should 
be  compensated.   [Ref.  37] 

The  whole  critical  incident  process  involves  a  con- 
tinuous flow  of  feedback  to  the  evaluated  employee. 

Although  the  critical  incident  process  by  its  very 
nature  should  lead  to  a  continuous  review  and  communication 
on  positive  and  negative  behavior,  that  is  not  always  the 
case.   Also  the  critical  incident  itself  is  often  a  subjec- 
tive evaluation  on  the  part  of  the  supervisor.   The  system 
should  focus  attention  on  actual  behavior  rather  than  on 
employee  traits.   Also  it  provides  an  opportunity  for  the 
employee  to  find  out  specifically  how  to  perform  if  that 
individual  wants  to  be  rated  higher  the  next  time.   This 
system  does  encourage  regular  feedback  since  the  supervisor 
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can  hardly  wait  to  the  end  of  a  period  to  provide  the  critical 
incident  feedback  [Ref .  38] .   One  of  the  major  factors  supporting 
this  system  is  that  the  critical  incident  process  takes  the 
surprise  out  of  the  annual  or  semi-annual  evaluation.   All 
those  incidents  viewed  by  the  supervisor  as  critical  would 
have  already  been  brought  to  the  attention  of  and  discussed 
with  the  subordinate.   This  technique  provides  more  regular 
feedback  to  the  subordinate  because  both  good  and  bad  inci- 
dents should  be  noted  continually.   Also  if  the  employee  feels 
unfairly  judged,  that  individual  may  appeal  the  criticism 
immediately  rather  than  waiting  a  long  period  of  time  [Ref.  37]. 

E.   SPECIFIC  EVALUATION  MODELS  APPLIED  TO  THE  MILITARY 

The  approaches  to  the  evaluation  process  discussed  thus 
far  have  been  general  in  nature.   However,  evaluation  proc- 
esses specific  to  military  environments  were  also  reviewed. 
Following  are  four  evaluation  approaches  to  the  military 
which  were  reviewed  in  detail. 

1 .   Army  Training  and  Evaluation  Program  (ARTEP) 

The  Army  Training  and  Evaluation  Program  (ARTEP)  for 
Infantry  Battalions  was  reviewed  for  this  study.   A  detailed 
brief  on  ARTE?  was  provided  by  the  G-3,  7th  Infantry  Division 
[Ref.  39 J  .   The  Army  Program  specifics  are  provided  for 
Infantry,  Airborne,  Air  Assault,  and  Ranger  units  in  one 
volume  [Ref.  40].   That  guide  is  further  supported  by  local 
guidance  at  each  command. 
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As  an  example  of  the  ARTEP, the  Infantry  ARTEP  is 
discussed.   The  Infantry  ARTEP  Evaluation  has  four  purposes: 
(other  references  exist  for  ARTEP  application  to  other  areas, 
such  as  Armor  and  Mechanized  units) : 

1.  Establish  infantry  unit  training  missions  with 
specified  tasks,  conditions,  and  standards  of 
performance  for  combat-critical  missions. 

2.  Under  simulated  combat  conditions,  train  and 
evaluate  the  ability  of  the  unit. 

3.  Evaluate  the  effectiveness  of  past  training  of  all 
echelons  of  the  Battalion. 

4.  Assess  future  training  needs.   [Ref.  40] 
The  evaluation  is  broken  into  three  phases: 

1.  Critique  of  Leaders 

2.  Evaluation  of  Whole  Battalion 

3.  Written  Report 

The  ARTEP  employs  the  following  philosophy: 

1.  The  evaluation  is  conducted  two  levels  down  (Division 
evaluates  Battalion) 

2.  Battalion  ARTEP ' s  are  scheduled  every  18  months. 

3.  Companies/Platoons/Squadrons  are  evaluated  annually. 

4.  Subunit  evaluators  focus  on  the  end  evaluation. 

5.  Evaluations  are  scheduled  in  sequential  progression 
(squad,  platoon,  company,  then  battalion) 

6.  Scheduled  preparation  time  is  five  weeks  for  a 
Battalion 
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7.  Fence  off  the  unit  for  two  weeks  following  the  exer- 
cise (no  other  requirements  are  placed  on  the  unit 
during  this  period) 

8.  Use  operating  force  tactics 

9.  Do  not  evaluate  during  other  training  or  field 
exercises.   [Ref.  39 J 

The  ARTEP  Evaluations  are  conducted  strictly  within 
the  divisional  unit.   The  Assistant  Division  Commander  is 
designated  "exercise  director, "  but  a  sister  battalion  is 
designated  as  the  force  to  provide  the  senior  evaluator, 
another  lieutenant  colonel  as  well  as  company  and  other 
evaluators.   The  ARTEP  is  viewed  as  a  training  evaluation. 

The  ARTEP  System  provides  a  combination  of  both  objec- 
tive and  subjective  feedback  to  the  evaluated  unit.   All  of 
the  tasks  evaluated  in  the  ARTEP  evaluation  are  reported  on 
a  Satisfactory/Unsatisfactory  basis.   All  unsatisfactory  or  not 
evaluated  ratings  must  be  explained  in  the  remarks.   This 
data  is  placed  on  a  standardized  data  collection  sheet.   A 
copy  is  provided  to  the  evaluated  unit  shortly  after  the 
exercise  is  completed.   Additionally  a  copy  of  the  data  sheet 
is  forwarded  (without  any  identifying  unit  data)  to  the  U.S. 
Army  Infantry  School,  Fort  Benning,  GA.   The  data  forwarded 
to  higher  headquarters  is  for  the  purpose  of  future  training 
development  efforts  and  for  use  in  improving  the  ARTEP  doc- 
trine, devices,  and  techniques  Army-wide  [Ref.  40].   The 
overall  evaluation  given  to  a  unit  is  a  Satisfactory/ 
Unsatisfactory.   No  percentile  scores  are  associated  with  or 
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computed  for  the  ARTEP  results.   Thus  no  comparison  is  made 
from  unit  to  unit  using  ARTEP  data  [Ref.  39]. 

In  addition  to  the  written  report,  a  continuous  review 
and  exchange  of  comments  between  the  evaluator  and  unit  is 
encouraged.   Shortly  after  the  ARTEP  is  completed,  an  oral 
debrief  is  held  for  all  key  personnel  of  the  battalion  and 
the  evaluators.   In  some  cases  the  detailed  (good)  evaluations 
of  units  to  the  squad  level  are  announced  to  all  of  the  troops 
at  formation.   This  technique  is  viewed  as  a  positive  motiva- 
tion for  the  small  units  and  their  leaders  [Ref.  39]. 
2 .   Funk  Conceptual  Model  of  Unit  Performance 

A  recent  study  by  Steven  Funk  [Ref.  13]  attempted  to 
model  combat  unit  effectiveness  for  the  U.S.  Army.   He  found 
many  of  the  same  difficulties  in  defining  effectiveness  that 
this  author  found  in  defining  readiness.   He  described  effec- 
tiveness as  follows: 

Individual  evaluators  determine  unit  effectiveness 
based  upon  some  explicit  or  intuitive  constraint  of 
what  units  are  supposed  to  do  and  how  that  is  achieved. 
Unit  effectiveness  is  determined  by  evaluating  both 
outcomes  and  processes,  and  is  determined  for  perform- 
ance on  all  tasks  presented  to  the  unit  whether  they 
are  combat  related  or  not.   [Ref.  13:  p.  19] 

He  interviewed  groups  of  officers  and  senior  enlisted, 
primarily  battalion  commanders,  company  commanders,  staff 
officers,  unit  officers,  and  first  sergeants.   He  also  re- 
viewed a  wide  array  of  existing  predictive  models  from  the 
Army,  Navy,  and  Air  Force.   He  identified  early  readiness 
studies  such  as  Army  Training  Study  (ARTS)  1977-1978,  which 
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concluded  that  readiness  should  be  viewed  from  a  perspective 
of  relationships  among  personnel,  weapons,  equipment,  re- 
sources, detractors,  and  incentives.   He  also  reviewed  the 
Navy's  1974  report  on  readiness  which  stated,  "Organizational 
effectiveness  implies  an  evaluative  or  judgmental  process 
against  an  expectation  or  standard"  [Ref.  13:  p.  11]. 

The  result  of  his  studies  concluded  that  military 
unit  effectiveness  is  far  more  readily  conceptualized  than 
measured  [Ref.  13].   Part  of  the  problem  of  measuring  mili- 
tary unit  effectiveness  is  operationally  defining  readiness 
in  a  peacetime  environment. 

Funk  referenced  Etzioni  who  noted  that  organizations 
attempt  to  achieve  a  balanced  distribution  of  resources 
across  needs  and  did  not  attempt  to  maximize  satisfaction  in 
one  area  [Ref.  13] .   Funk  concluded  that  military  units  in 
peacetime  have  great  difficulty  in  maximizing  effectiveness 
or  readiness  because  they  really  face  not  one  objective,  but 
a  whole  array  of  competing  priorities  all  calling  for  their 
resources.   His  studies  resulted  in  the  proposed  conceptual 
model  in  Figure  3.3. 

This  model  is  referred  to  as  the  Unit  Performance 
Systems  model.   It  describes  behavior  as  the  interrelation- 
ships of  technology,  formal  and  informal  unit  structure, 
perceived  unit  requirements  and  priorities,  available  re- 
sources, unit  climate,  unit  process,  cause  of  evaluation  of 
unit  actions,  and  the  unit  operating  environment  [Ref.  11]. 
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Figure  3.3    Unit  Performance  Systems   Model 
[Ref.  13  =   p.  38 
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3 .   Hayes  et  al.,  Statistical  Model  of  Combat  Effectiveness 
In  1977,  a  cross-sectional  study  for  the  Defense  Ad- 
vanced Research  Projects  Agency  was  conducted.   An  effort  was 
made  to  identify  through  detailed  statistical  methods  key 
factors  which  contributed  to  levels  of  success  in  22  combat 
engagements  ranging  from  World  War  II  to  Vietnam.   The  study 
was  aimed  at  determining  the  effectiveness  of  infantry  battalions 
based  on  the  judgment  of  experienced  Marine  Corps  officers  in- 
volved in  these  engagements.   Multi-variate  statistical  analy- 
sis was  used  to  examine  the  data  results.   The  study  revealed 
that  adaptive  behavior  by  units,  that  is  reaction  to  the  com- 
bat environment,  was  the  single  most  important  discriminator 
between  successful  and  unsuccessful  performance.   They  also 
identified  three  types  of  activity  which  appeared  to  be 
closely  related  to  mission  accomplishment:   command  and 
planning  (strongly  related) ,  supporting  fires  (moderately 
associated) ,  and  coordination  function  (relatively  weak  and 
appeared  to  require  effective  command  and  planning  before  it 
made  a  difference)  [Ref.  18]. 

This  study  defined  "combat  effectiveness"  as 

. . .the  ability  of  a  unit  to  accomplish  a  military 
mission.  As  such,  combat  effectiveness  refers  to 
performance  in  a  hostile  environment.   [Ref.  18:  pp.  1-2] 

As  such  the  focus  on  the  effectiveness  of  a  military  unit  is 

placed  on  outcomes  and  effectiveness  and  can  only  be  measured 

by  mission  accomplishment.   The  infantry  battalion  was  seen 

as  not  operating  in  a  vacuum,  but  rather  in  a  world  of 
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obstacles  and  constraints.   Thus  the  adaptability  was  an 
important  element  in  performance.   Besides  adaptability,  Hayes 
identified  a  number  of  key  elements  which  included: 

•maneuver  during  the  action 

•preparatory  air,  artillery,  and  naval  gunfire 

•communications 

•quality  of  planning  and  information  as  well  as  others. 
[Ref.  18:  pp.  1-18] 

This  study  was  done  during  the  period  when  MCCRES  was 
being  developed.   One  important  observation  made  here  with 
regard  to  feedback  was  that,  although  MCCRES  is  comprehensive 
and  standardized,  a  great  deal  of  information  is  currently 
passing  through  the  hands  of  the  evaluators  that  could  be 
extremely  valuable  to  the  evaluated  units  [Ref.  18]. 

4 .   Sassone  Economic  Model  of  Training  Effectiveness 

A  fourth  approach  to  effectiveness  is  taken  by 
Sassone,  who  has  taken  an  economic  view  which  is  different 
from  the  traditional  economic  approach  to  evaluating  military 
training  programs.   The  traditional  approach  is  to  evaluate 
the  training  on  its  projected  costs  and  benefits.   The  Sassone 
approach  requires  that  any  new  training  program  be  compared 
directly  with  existing  programs  that  it  will  replace  or 
amplify.   Traditionally  equipment  and  training  are  judged 
simply  on  their  ability  to  function  for  the  intended  purpose. 
The  training  programs  are  particularly  difficult  to  evaluate 
because  of  the  necessity  of  measuring  the  impact  of  the  train- 
ing on  the  trainee.   In  the  private  sector  training  programs 
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are  generally  judged  on  their  effectiveness  by  the  change  in 
lifetime  earnings  of  the  trainee  after  the  program  [Ref.  41]. 
If  the  cost  of  the  training  program  is  less  than  the  lifetime 
increase  in  wages,  then  this  program  is  judged  to  be  economi- 
cally worthwhile.   However,  training  programs  in  the  mili- 
tary cannot  be  judged  this  same  way,  because  there  is  currently 
no  way  to  measure  their  contribution  toward  readiness,  sus- 
tainability,  or  effectiveness  since  these  are  not  commodities 
or  traded  in  the  market  place  [Ref.  41] . 

Sassone's  methodology  requires  that  when  a  new  type 
of  training  for  an  individual  or  unit  is  developed,  it  must 
be  stated  in  specific  terms  to  what  extent  it  substitutes  for 
existing  programs.   The  constraint  in  this  case  is  the  train- 
ing budget  for  the  specific  type  of  training.   There  are 
three  steps  in  this  methodology  [Ref.  41]. 

(1)  An  equation  relating  training  inputs  and  outputs  must 
be  developed.   This  requires  a  comparison  of  the  two  types 
of  training  under  consideration  (see  Figure  3.4).   The 
figure  shows  different  levels  of  effectiveness  which  can  be 
achieved  with  various  combinations  of  the  two  types  of  train- 
ing.  The  curve  shows  all  the  levels  of  effectiveness  which 
can  be  achieved  with  the  various  combinations.   The  shape  of 
the  curves  describe  the  substitute  ability  of  one  training 
type  for  the  other. 

(2)  Relationship  of  cost  and  use  data  must  be  obtained. 
There  are  three  types  of  costs  involved:   front  end  costs  for 
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the  training  program  under  evaluation;  operational  costs,  and 
training  budgets  for  the  training  manager  of  the  program. 

(3)   The  last  step  requires  the  development  of  a  relation- 
ship between  training  and  effectiveness. 

Where  potentially  greater  effectiveness  is  available 
at  greater  cost,  the  issue  is  simply  whether  the 
greater  effectiveness  is  worth  the  greater  cost. 
[Ref.  41:  p.  41] 

Sassone  fails  to  specify  how  the  relation  of  training  and 

effectiveness  is  obtained.   It  is  assumed  in  this  study  that 

the  relationship  is  identified  by  some  evaluation  method  such 

as  MCCRES .   Measures  such  as  those  generalized  by  ARTEP  allow 

for  no  relative  measure  of  effectiveness  which  is  required  for 

this  type  of  comparison. 

The  cost-effectiveness  equation  is  then  developed 
by  optimum  use  of  available  training  resources  at  each  level 
of  the  training  budget.   The  term  "optimum"  in  Sassone* s 
proposal  is  one  that  seems  to  be  in  conflict  with  other 
theorists.   Funk  [Ref.  13]  for  instance  indicates  that  many 
competing  priorities  take  resources  from  the  unit,  thus 
precluding  optimality  of  resource  application. 

Figure  3.5  represents  a  relation  between  cost  and  a 
sample  program.   Figure  3.5  is  described  by  Sassone  as  starting 
at  a  minimal  level,  indicating  that  some  level  of  effective- 
ness would  exist  even  without  training.   The  curve  initially 
rises  rapidly  but  starts  to  level  off  as  maximum  effectiveness 
is  reached.   As  better  training  programs  are  conceived,  the 
middle  of  this  curve  would  shift  up.   If  some  new  program  were 
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a  perfect  substitute  of  an  existing  one,  the  new  curve  would 
overlay  the  existing  one.   Upward  shifts  in  the  curve  can  be 
caused  by  a  greater  level  of  effectiveness  achieved  for  a 
lower  cost.   The  Figure  3.5  shows  how  the  military  value  of 
various  training  programs  can  be  compared  with  other  or  exist- 
ing programs.   Additionally,  one  must  consider  opportunity 
costs,  funds  used  for  research  and  development  of  new  pro- 
grams which  the  service  could  otherwise  have  devoted  to 
existing  programs  [Ref.  41]  . 

The  military  value  of  the  new  training  program  is  repre- 
sented by  the  increment  in  the  current  training  budget 
needed  to  increase  effectiveness  by  the  same  amount 
as  the  increase  in  effectiveness  associated  with  the 
new  program.   [Ref.  41:  p.  42] 

The  opportunity  cost  of  a  new  military  training  program  is 

determined  by  spreading  the  front  end  costs  equally  over 

the  units  trained  per  year  and  the  expected  life  time  of  the 

program  applying  the  appropriate  discount  rate.   Figure  3.5 

is  used  to  compare  the  military  value  of  a  training  program 

by  using  the  cost-effectiveness  ratio,  if  the  military  value 

is  less  than  the  opportunity  cost,  then  the  new  program  would 

not  be  considered  economically  feasible  [Ref.  41], 

5 .   Summary 

This  section  reviewed  four  approaches  to  evaluation 

that  have  recently  been  developed  for  applications  to  the 

military.   The  ARTEP  employed  [Ref.  40]  by  the  Army  provided 

a  process  of  evaluation  which  has  much  in  common  with  MCCRES 

especially  in  its  application  to  similar  organizations.   The 

conceptual  model  by  Funk  [Ref.  13]  provided  a  view  not  only 
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of  the  elements  involved  in  training,  but  also  of  some  of 
the  barriers  and  constraints  that  units  must  face  in  a  peace- 
time environment.   The  results  of  the  Hayes  et  al.,  Statistical 
Model  [Ref.  18]  were  valuable  in  that  specific  characteris- 
tics were  identified  that  led  to  success  in  actual  combat 
engagements.   Finally  the  Sassone,  Economic  Training  Model 
was  presented  [Ref.  41]  which  proposed  some  new  thoughts  on 
the  subject  of  evaluating  military  training  programs.   The 
ideas  brought  forth  in  these  studies  are  of  use  in  the  examina- 
tion of  MCCRES  in  that  they  provide  a  rich  background  against 
which  MCCRES  can  be  viewed. 

F .   FEEDBACK 

...studies  have  shown  that  accurate  feedback  about 
quality  of  work  is  a  strong  stimulus  to  good  perform- 
ance. .  .People  work  better  when  they  know  how  well  they 
are  doing  in  relation  to  some  meaningful  standard. 
[Ref.  42:  p.  174] 

This  final  section  of  the  chapter  discusses  feedback. 
Feedback  is  variously  described  by  different  writers  as  feed- 
back [Refs.  27,29,37,43],  feedback  control  [Ref.  16J , 
communications  [Ref.  34]  and  further  as  dissemination, 
communications,  and  utilization  [Ref.  35].   Whatever  the 
feedback  is  called, it  is  common  to  all  of  the  models  dis- 
cussed.  Feedback  is  a  necessary  part  of  all  evaluation 
systems . 

Control  to  be  effective,  requires  timely,  accurate,  and 
dependable  indicators  of  effectiveness.   Feedback  should 
provide  information  that  is  adequate  to  suggest  appro- 
priate action.   To  that  end,  reports  must  point  to 
significant  developments,  as  distinguished  from  what  is 
normal,  usual,  and  to  be  expected.   [Ref.  32:  p.  504] 
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Flamholtz  [Ref.  44]  has  presented  a  typical  model, 
Figure  3. 6, of  an  organizational  control  system  which  places 
emphasis  on  the  feedback  loop.   He  describes  feedback  as  a 
necessary  managerial  function  of  the  control  systems  which 
identifies  problems  [Ref.  44 J . 

This  model  provides  an  example  of  how  the  essential  feed- 
back loop  functions  in  an  organizational  control  or  evaluation 
system.   The  feedback  loop  provides  an  interactive  process 
that  allows  for  routine  recycling  of  the  results  of  a  compari- 
son of  the  observed  performance  to  the  measurement  system. 
The  essential  element  to  observe  here  is  that  feedback  is  a 
standard  fixture  of  the  evaluation  system.   All  systems  viewed 
that  provide  a  diagrammatic  model  include  a  feedback 
mechanism.   In  addition  Anderson  and  Ball  and  the  ARTEP 
provided  detailed  structures  for  feedback. 

G.   SUMMARY 

This  chapter  has  covered  the  topics  of  management,  manage- 
ment control,  evaluation,  and  feedback.   These  topics  can  be 
seen  as  moving  from  the  general  to  the  specific.   The  bound- 
aries between  these  topics  are  not  clear  cut.   There  is  a 
great  deal  of  overlap  in  the  fields  of  management,  organiza- 
tional control,  evaluation,  and  feedback.   This  is  particu- 
larly true  when  they  are  viewed  from  different  disciplines. 
For  instance,  much  of  what  is  classified  in  education  as 
evaluation,  would  be  classified  under  control  in  organizational 
or  managerial  theory.   A  similar  situation  exists  with  regard 
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Figure  3.6     Model  of  Organizational  Control  System. 
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to  terminology  for  feedback.   Theorists  in  the  same  and 
different  disciplines  often  use  various  terms  to  describe 
feedback. 

Feedback  was  discussed  with  emphasis  on  relating  the 
many  different  terms  used  to  describe  feedback.   In  addition 
the  feedback  loop  which  is  common  to  all  models  reviewed 
has  been  discussed.   The  theory  of  feedback  is  essential  to 
this  study  and  the  many  ideas  and  structures  placed  on  it 
will  give  much  insight  when  applied  to  MCCRES.   Certain  ele- 
ments such  as:   brevity  and  clarity,  timeliness,  interim 
products  and  reports,  and  responsiveness  [Ref.  35]  have  almost 
universal  application  to  feedback  theory.   The  MBO ,  Evaluation 
and  Authority,  Critical  Incident,  and  ARTEP  all  provide  a 
wealth  of  insight  on  how  feedback  techniques  should  be 
applied. 

The  next  chapter  provides  a  detailed  overview  of  the 
MCCRES  System. 
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IV.   MCCKES 

In  the  last  chapter  a  detailed  examination  was  made  of 
management  control,  four  general  evaluation  systems,  and  four 
specific  evaluation  systems  which  apply  to  the  military. 
This  should  provide  the  reader  with  a  background  against 
which  to  view  MCCRES.   In  this  chapter  attention  is  given  in 
detail  to  the  MCCRES  itself,  its  development,  explanation  of 
the  system's  mission  performance  standards,  and  the  evalua- 
tion process  applied  by  MCCRES  as  well  as  the  reports  and 
feedback  designed  into  the  current  system.   The  goal  of  this 
chapter  is  to  provide  the  reader  with  a  general  understanding 
of  how  the  MCCRES  is  applied,  based  on  the  most  current 
Marine  Corps  directives. 

The  purpose  of  the  Marine  Corps  Combat  Readiness  Evalua- 
tion System  (MCCRES)  is  to  provide  a  timely  and  accurate 
evaluation  of  readiness  of  Fleet  Marine  Forces,  includ- 
ing reserve  units,  to  accomplish  assigned  missions. 
[Ref.  1:  p.  1-A-l] 

The  formal  MCCRES  must  be  given  at  least  once  every  two  years 

for  all  FMF  units  for  which  performance  standards  have  been 

written. 

A.   DEVELOPMENT  AND  STRUCTURE 

The  development  of  the  MCCRES  was  begun  with  a  review  of 
the  various  combat  readiness  measurements  then  currently  in 
use.   Many  of  these  consisted  of  individual,  as  well  as  unit, 
evaluations  and  were  in  the  form  of  inspections  which  stressed 
appearance  more  than  combat  proficiency  [Ref.  1] . 
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The  MCCRES  was  structured  to  access  the  ability  of  all 
Marine  Corps  Combat  units:   air  and  ground  combat,  combat 
support,  and  combat  service  support.   Specifically  it  was 
designed  to  provide: 

-Performance  standards  based  on  assigned  missions 

-A  standardized  evaluation  process 

-A  standardized  reporting  system 

-Feedback  to  units  indicating  strengths  and  weaknesses. 
[Ref.  1:  p.  l-A-2] 

The  MCCRES  was  designed  to  provide  a  detailed  analysis 
of  the  units'  operational  capabilities  and  to  specifically 
identify  strengths  and  weaknesses  through  comparison  with 
doctrine  under  simulated  combat  conditions.   The  doctrine  is 
embodied  in  a  checklist  of  requirements.   The  hierarchy  of 
these  qualities  is  identified  from  general  to  specific  as 
Mission  Performance  Standards  (MPS) ,  each  of  which  consists 
of  three  parts:   Tasks,  Conditions,  and  Requirements.   The 
MPS  set  the  Commandant's  acceptable  standards  for  tactical 
performance  throughout  the  Marine  Corps.   Based  on  compari- 
son of  the  unit's  performance  to  the  MPS  under  simulated 
conditions,  a  determination  is  made  of  "Combat  Ready/Not 
Combat  Ready."   The  MCCRES  is  designed  for  both  formal  and 
informal  evaluation.   The  informal  evaluation  can  be  pro- 
vided by  the  unit  itself  measuring  its  performance  standards. 
The  formal  evaluation  is  accomplished  by  higher-level  commands 
normally  evaluating  a  selection  of  MPS.   For  the  formal 
evaluation  certain  MPS  are  required  in  all  evaluations  and 
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others  are  selected  by  the  organization  directing  the  evalua- 
tion based  on  detailed  guidance  in  Ref.  1.   A  cross  section 
of  MPS's  are  used  to  provide  a  statement  of  the  organiza- 
tion's operational  readiness. 

The  evaluators  employed  in  the  formal  system  must  be  ex- 
perienced, capable  personnel.   The  overall  system  is  designed 
to  identify  specific  operational  deficiencies  and  the  nature 
of  any  latent  or  potential  problems.   The  MCCRES  cycle  is 
described  in  Figure  4.1. 

B.   MISSION  PERFORMANCE  STANDARDS  (MPS) 

In  building  the  MCCRES  framework  it  was  decided  that  the 
evaluation  would  be  structured  around  a  set  of  Mission  Per- 
formance Standards  that  establish,  on  a  Marine  Corps-wide 
basis,  acceptable  operational  performance.   Derivation  of 
these  standards  was  accomplished  by  using  five  interrelated 
concepts  concerning  determination  of  combat  readiness.   The 
concepts  are: 

-  Standards  must  be  objective 

-  Standards  must  define  for  the  evaluator  what  quality 
means 

-  Standards  must  be  based  on  published  doctrine 

-  Standards  must  involve  the  performance  of  individual 
Marines  in  evaluation  of  unit  combat  readiness 

-  Standards  must  be  simple  for  the  evaluator.   [Ref.  1] 
The  MPS  are  defined  in  detail  in  Volumes  II  through  VIII 

of  Marine  Corps  Order  3501.2.   Each  of  these  volumes  pertains 
to  a  specific  type  of  unit  to  be  evaluated,  for  instance 
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Volume  II  pertains  to  Infantry  and  Volume  III  to  Rotary  Wing 
Observation  Squadrons.   In  this  way  the  volume  lists  the 
MPS's  involved  for  each  unique  type  of  unit. 

Each  MPS  is  further  broken  down  into  three  major  parts: 

-The  Task  to  be  performed 

-The  Condition  under  which  the  task  is  to  be  performed 

-The  Requirements  which  must  be  accomplished  to  success- 
fully fulfill  the  task.   [Ref.  1:  p.  I-B4J 

Examples  of  the  MPS,  Tasks,  Conditions,  and  Requirements 
are  provided  in  Figure  4.2. 

To  assist  the  Evaluator  when  the  Requirement  does  not 
completely  define  the  quality  of  performance  necessary,  Key 
Indicators  (KI)  are  provided.   These  KI's  are  defined  to 
clarify  and  explain  the  requirements  [Ref.  1]  .   Samples  of 
Key  Indicators  are  presented  in  Figure  4.3. 

For  simplicity  and  to  reduce  the  need  for  subjective  judg- 
ments, all  MPS's  were  designed  to  permit  the  Evaluator  only 
three  possible  outcomes  for  each  requirement:   Yes/No/Not 
Applicable.   Those  MPS's  dealing  with  the  performance  of  indi- 
vidual Marines,  Exercise  of  Command  and  Control,  and  Fire 
Support  Coordination  must  be  used  during  every  evaluation. 
The  reason  that  all  MPS's  are  not  employed  during  each 
evaluation  is  a  limitation  of  time  and  resources  [Ref.  1J . 

In  addition  to  the  evaluation  of  MPS's  which  use  a  simu- 
lated combat  environment  against  an  aggressor  force,  a  series 
of  MCCRES  Standard  Performance  Tests  (SPT)  have  also  been 
developed.   Examples  of  these  types  of  proficiency  tests  are: 
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MISSION  PERFORMANCE  STANDARD  2B. 3— MOVEMENT  TO  CONTACT 


CONDITIONS        REQUIREMENTS 


Y  N    COMMENTS 


Operation  order  is 
provided  directing  a 
tactical  movement  to 
area  of  anticipated 
contact  with  enemy. 

Route  selection  is  a 
tactical  decision  of 
unit  commander. 

Minimum  preparation 
time  for  proper 
evaluation  of  this 
task  is  three  hours 
after  delivery  of 
the  order  to  the 
unit. 


Adopted  from  (Ref.  1: 
p.  1-B-5) 


2B. 3. 1.1—  Receipt  of  order 

acknowledged  to  higher 
hq. 

2B. 3. 1.2— Warning  order  issued  to 
all  suPordinate  units 
within  30  minutes  (Kl). 

23.3.1.3— Staff  coordination 

IAW  FMFM3-1;  Emphasis: 
Intelligence  collection, 
fire  support  planning, 
and  security  (Kl). 
(Key  Indicator) 

2B.3.1.4 — All  echelons  prepare  for 
move;  inspect  material, 
pack  equipment,  and  iden- 
tify specialized 
requirements. 

2B. 3. 1.5— Weapons  test  fired  if 
situation  permits. 

2B.3. 1.6— Communications  checks  com- 
pleted and  communication 
security  materials  issued. 

2B. 3. 1.7— Movement  plan  formulated  (Kl 

2E. 3. 1.8— Movement  order  issued  (Kl). 

2B.3.1.9 — Readiness  for  movement  re- 
ported to  higher  hq. 


Figure  4.2.  Sample  of  Mission  Performance  Standards 
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WARNING  ORDER 

MUST  INCLUDE: 

General  information  on  the  situation. 

Units  to  make  the  move  and  anticipated  sequence. 

Anticipated  time  of  move. 

Anticipated  route  and  destination. 

STAFF  COORDINATION 

MUST  SPECIFICALLY  ADDRESS: 

Route  reconnaissance— map,  physical,  or  aerial  photo. 
Support  needed  from  higher  hq — air,  vehicle,  or  firepower. 
Fire  support  coverage  throughout  movement. 
Cover  and  concealment  available  enroute. 
Logistic  aspects  affected  by  location  change. 

BASIC  PLAN 

MUST  BE  BASED  ON: 

Movement  at  speed  desired  by  higher  hq. 
Control  of  ail  elements  during  move. 
Security  of  all  movement  serials. 
Known  enemy  capabilities. 

MOVEMENT  ORDER 

MUST  CONTAIN: 

Clearly  stated  mission. 

Definition  of  all  control  measures  to  be  used:  check  points,  phase  lines,  march  objectives,  etc. 

Specific  missions  for  attached  and/or  support  elements:  tanks,  engineers,  TOW,  etc. 

Identification  of  initial  point  from  which  move  will  begin. 

Assignment  to  march  serials.  _  _ 

Time  of  departure. 

Clear  identification  of  available  outside  support. 

Detailed  security  procedures. 

As  much  information  on  threat  as  is  available: 
Emphasis  on  specialized  weaponry  tnat  can  affect  the  move:  ATGM,  artillery,  air,  etc. 

Source:     Adopted  from  (Ref.  1.P:  p.  I-B-6) 

Figure  4.3     Sample  of  Key  Indicators  for  Evaluator 
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Foot  Mobility  Test,  Dragon  Gunner's  Test,  Engineer  Route 
Reconnaissance  Test,  and  Aircraft  Recognition  Test.   The 
SPT's  are  used  to  support  the  MPS ' s  and  are  tested  objectively 
prior  to  the  MCCRES  operational/readiness  test.   The  SPT  is 
formated  exactly  as  the  MPS;  however,  a  'Yes'  must  be  achieved 
in  each  area  before  the  MCCRES  is  continued.   Any  unit  not 
found  proficient  in  the  SPT  portion  of  MCCRES  will  not  be 
tested  on  the  tactical  portion. 

C.   PERSONNEL  ASSIGNMENT  FOR  THE  EVALUATION 

The  key  to  successful  application  of  each  MCCRES  is  the 
evaluation  process.   Selection  and  training  of  evaluators  is 
at  the  very  heart  of  the  evaluation  process.   Credibility  of 
the  evaluators  is  essential  to  the  success  of  MCCRES.   The 
evaluators  selected  are  to  have  the  requisite  skills  and 
recent  experience  as  to  preclude  the  need  for  a  long,  detailed 
school  for  the  evaluators.   However,  as  needed  and  as  time 
allows,  training  is  provided  to  evaluators.   Since  all  evalu- 
ation systems  require  some  judgment  on  the  part  of  the 
evaluators,  and  MCCRES  is  no  exception,  sound  judgment  is  a 
prerequisite  for  each  evaluator  chosen. 

The  evaluation  is  structured  with  the  Commanding  General's 
Fleet  Marine  Force  Pacific  (FMFPAC)  and  Fleet  Marine  Force 
Atlantic  (FMFLANT)  as  the  Evaluation/Exercise  Commander (s) 
(EC)  for  the  initiation  and  conduct  of  all  formal  MCCRES 
Evaluations.   The  responsibilities  of  the  Evaluation/Exercise 
Commanders  are  outlined  in  detail  in  Ref.  1. 
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1.  Evaluation/Exercise  Director  (ED) — is  designated  by  the 
Exercise  Commander  (EC)  and  is  responsible  to  prepare 
for,  conduct,  and  report  the  formal  MCCRES  evalua- 
tions.  Normally  the  Commanding  General  of  the  respec- 
tive Marine  Division  or  Wing  whose  subunit  is  to  be 
evaluated  is  designated  ED. 

2.  Tactical  Exercise  Controller  (TEC) — is  designated  by 
the  ED,  along  with  an  appointed  staff,  to  serve  as  the 
control  agency  for  conducting  the  exercise.   The  TEC 
compiles  and  analyzes  the  results  of  the  evaluation. 
The  TEC  is  also  responsible  for  the  detailed  training 
of  the  evaluators  as  well  as  development  of  a  detailed 
exercise  scenario.   At  the  end  of  the  evaluation  a 
formal  report  is  prepared  for  the  ED  describing  the 
combat  readiness  of  the  evaluated  unit.   A  detailed 
critique  should  also  be  conducted  for  all  involved  in 
the  evaluation.   In  the  critique  the  results  of  the 
evaluated  MPS  are  to  be  highlighted  with  respective 
strengths  and  weaknesses. 

3.  Evaluators — should  be  prepared  for  the  role  through 
successful  past  professional  experiences  as  well  as 
through  any  detailed  school  provided.   The  MCCRES 
evaluators  have  three  roles:   exercise  controller,  umpire 
and  performance  evaluator.   As  a  performance  evaluator, 
they  apply  the  detailed  MPS ' s  contained  in  the  appro- 
priate volume  for  the  unit  and  they  are  evaluating  and 
actually  make  the  'Yes/No/Not  Applicable'  determinations 
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for  individual  tasks.   The  evaluators  must  possess  a 
complete  and  thorough  understanding  of  the  Mission 
Performance  Standards  being  evaluated.   The 
evaluators  must  make  any  notes  needed  on  an  Evaluator 
Work  Sheet  that  are  necessary  to  support  judgment 
during  the  process.   [Ref.  1] 

4.  Senior  Evaluator — determined  by  rank.   Evaluator  for 
the  unit  evaluated  compiles  the  data  sheets  from  all 
evaluators  and  should  conduct  a  post  exercise  wrap  up. 
Any  questions  or  conflicts  should  be  resolved  at  that 
time.   The  process  must  result  in  the  senior  evaluator 
being  assured  that  his  data  provides  an  accurate  reflec- 
tion of  the  overall  unit  performance.   The  senior 
evaluator  then  makes  the  determination  of  the  overall 
unit  evaluation  of  "Combat  Ready/Not  Combat  Ready." 

The  senior  evaluator  provides  the  data  sheets  to  TEC 
who  compiles  the  detailed  analysis  and  presents  it  to 
the  evaluation/exercise  director  [Ref.  1]. 

5.  Evaluation  Staffing — is  the  responsibility  of  the  ED 
for  both  selecting  and  training  all  evaluators  prior 
to  a  MCCRES .   It  is  desirable  to  obtain  evaluators 
from  adjacent  commands,  not  directly  related  to  the 
organization  being  evaluated.   A  more  complete  discussion 
of  providing  the  evaluators  can  be  found  in  Ref.  6. 

D.   COMPUTER  ADAPTED  MCCRES 

The  Marine  Corps  Combat  Readiness  System  Software  Appli- 
cation (MCCRESSA) ,  is  designed  to  furnish  all  organizations 
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involved  with  MCCRES  a  means  of  assessing  and  analyzing  data 

pertaining  to  MCCRES  evaluations.   Specifically  it  provides 

the  following: 

A  list  of  mission  performance  standards,  tasks,  and 
requirements . 

...the  chief  evaluator  with  a  rapid  overview  and  selected 
analysis  of  the  units  evaluation. 

Identifies  unit  readiness  based  upon  mission  performance 
standards  tested  within  the  applied  scenario. 

Provides  unit  commanders  a  list  of  unit  readiness  defi- 
ciencies to  initiate  corrective  action  or  obtain 
higher  command  level  assistance. 

Reviews  unit  evaluations  to  determine  validity  of  mission 
performance  standards  and  doctrine. 

Reviews  unit's  evaluations  to  determine  limitation/inade- 
quacies in  officer  education  programs.   [Ref.  1:  p.  I-E2] 

All  of  the  formal  evaluation  reports  are  input  to  the 

Headquarters  Marine  Corps  data  base  [Ref.  1] . 

E.   REPORTS 

At  the  end  of  a  MCCRES,  after  the  Evaluator  Data  Sheets 
have  been  compiled  for  all  units  evaluated,  the  TEC  and 
senior  evaluators  should  review  the  sheets  and  make  a  deter- 
mination of  "Combat  Ready/Not  Combat  Ready"  for  each  unit 
evaluated.   This  recommendation  is  based  on  an  initial  review 
of  "Yes's/No's"  for  all  requirements,  taking  into  account 
any  tasks  or  MPS ' s  that  were  "demand  elements  that  must  be 
judged  Yes  for  an  overall  'Combat  Ready1  evaluation  to  be 
given . " 
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1 .   Feedback 

"The  primary  purpose  of  MCCRES  reports  is  to  provide 
the  feedback  necessary  for  commanders  to  initiate  corrective 
action  that  will  improve  combat  readiness"  [Ref.  1:  p.  1-D1] . 
The  results  should  be  reflected  in  improved  training 
objectives  and  may  also  affect  resource  allocation  of  per- 
sonnel, equipment,  or  logistics  support.   To  assist  in  the 
reallocation  of  critical  resources,  a  detailed  report  is 
provided  through  the  chain  of  command  to  the  unit  evaluated. 
This  report  provides  a  short  subjective  comment  as  well  as 
percentile  scores  and  weighting  for  each  Section,  MPS,  Task, 
and  Requirement  evaluated.   (See  Appendix  A — Sample  MCCRESSA 
printout  for  one  MPS . ) 

Additionally  a  report  is  provided  in  message  format 
to  the  Commandant  of  the  Marine  Corps  (CMC)  within  10  days 
following  the  end  of  an  evaluation  indicating  the  overall 
evaluation  "Combat  Ready /Not  Combat  Ready"  and  unit  identifi- 
cation.  Within  30  days  a  detailed  report  of  the  MCCRESSA 
printout  is  provided  to  the  CMC.   The  report  should  include 
comments  or  recommendations  for  improvement  or  revisions  to 
the  MCCRES.   The  purpose  of  the  follow  up  report  is  to  allow 
CMC  to:   provide  assistance  in  the  review  of  doctrine,  tactics 
techniques,  education  and  training  programs,  and  validation 
of  MCCRES  elements .   This  report  also  serves  to  highlight 
trends  and  repeated  deficiencies,  and  permit  analysis  for 
corrective  action  to  take  place  [Ref.  1] . 
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2 .   Policy 

Based  on  Marine  Corps  policy,  MCCRES  reports  are  not 
designed  to  be  used  to  compare  the  combat  readiness  of  vari- 
ous units.   This  is  a  result  of  differences  in  unit  type, 
services,  environmental  conditions,  and  resource  allocations. 
The  evaluation  indexes  on  MCCRES  reports  are  provided  to 
indicate  an  approximate  status  of  the  unit's  combat  readiness 
during  a  particular  evaluation  [Ref .  1] .   It  can  be  said  that 
the  MCCRES  provides  a  snap  shot  of  the  unit  at  the  time  of 
the  evaluation. 

F.   SUMMARY 

This  chapter  presented  an  overview  of  the  MCCRES  including 
a  discussion  of  the  Development  and  Structure,  Mission  Per- 
formance Standards,  Personnel  Assignment  for  the  Evaluation, 
Computer  Adapted  MCCRES  and  Reports .   The  purpose  was  to 
provide  the  reader  with  a  general  background  of  how  the 
MCCRES  is  intended  to  be  implemented  throughout  the  Marine 
Corps,  based  on  systems  design  and  policies. 
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V.   METHOD  OF  ANALYSIS 

The  research  method  used  had  three  distinct  phases. 
First, a  general  literature  review  was  conducted.   Second, a 
specific  literature  review  of  MCCRES  and  related  defense  pub- 
lications was  undertaken.   Third, a  detailed  field  study  was 
made  to  obtain  first  hand  information  from  those  involved 
with  the  development,  application,  and  oversite  of  MCCRES. 

A.  AUDIENCE 

Anderson  and  Ball  discussed  [Ref.  35]  the  importance  of 
identifying  the  audience  when  researching  and  distributing 
results.   To  this  end  the  primary  audience  for  this  study 
are  the  planners  who  conduct  periodic  reviews  and  revisions 
of  MCCRES  doctrine,  that  is,  the  Readiness  Branch  at  KQMC. 
Additionally  the  audience  should  include  all  those  involved 
with  MCCRES  or  who  will  be  involved  with  MCCRES,  as  well  as 
commanders,  evaluators,  planners  in  organizations  and  others 
who  would  benefit  from  understanding  the  findings  presented 
herein. 

B.  GENERAL  LITERATURE  REVIEW 

The  results  of  the  general  literature  review  are  presented 
in  Chapter  III.   A  background  in  management  control,  general 
evaluation  systems  as  well  as  specific  examples  of  evaluation 
systems  applied  to  the  military  provided  a  backdrop  against 
which  MCCRES  can  be  viewed.   It  is  evident  from  Chapter  IV 
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that  MCCRES  has  incorporated  into  it  many  of  the  positive 
aspects  of  general  evaluation  systems  theory  [Refs.  5,7,22, 
27,28,34,35] . 

C.  SPECIFIC  LITERATURE  REVIEW 

The  specific  literature  review  consisted  of  articles  on 
MCCRES  appearing  in  the  various  defense  publications  [Refs.  2, 
7,11,14,33,39,  and  40]  as  well  as  the  technical  reports  and 
articles  available  on  MCCRES  and  other  military  training  and 
readiness  evaluation  systems  [Refs.  4,8,12,13,18,40].   This 
specific  literature  review  not  only  provided  a  basis  for 
understanding  the  MCCRES  from  the  standpoint  of  its  proponents 
and  detractors,  but  it  also  provided  some  understanding  of 
alternatives  to  MCCRES.   The  detailed  reports  made  available 
by  George  Washington  University  were  invaluable  in  understand- 
ing the  background  and  initial  development  of  the  MCCRES 
[Refs.  3,4,11]. 

D.  FIELD  STUDY 

The  field  study  was  conducted  to  obtain  first  hand  infor- 
mation on  MCCRES.   Personal  interviews,  telephone  interviews, 
and  a  visit  to  the  Readiness  Branch  HQMC,  George  Washington 
University,  and  to  the  Army's  7th  Infantry  Division  were  em- 
ployed to  that  end.   The  interviews  were  semi-structured. 
The  stated  purpose  of  the  interviews  was  not  just  to  answer 
the  specific  questions  posed,  but  to  stimulate  thought  and 
identify  as  many  ideas  as  possible  on  how  the  MCCRES  could 
be  improved.   Although  the  scope  of  this  project  was  limited 
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by  its  primary  emphasis  on  feedback,  other  valuable  informa- 
tion was  elicited. 
1 .   Sample 

The  sample  group  chosen  consisted  of  those  who  had 
the  most  experience  and  most  vested  interest  in  the  MCCRES 
feedback.   Because  the  MCCRES  evaluations  are  most  often 
given  to  infantry  battalions  and  aviation  squadrons,  these 
were  identified  as  the  primary  target  group.   Battalion  and 
squadron  commanders,  senior  evaluators ,  and  division/squadron 
MCCRES  officers  were  identified  as  the  primary  sources  of 
information  since  they  have  the  most  detailed  involvement  with 
the  MCCRES  feedback.   Upon  interviewing  the  sample  it  was 
also  found  that  many  of  the  officers  had  been  involved  in 
several  MCCRES * s  and  in  different  capacities.   Thus  many  had 
experience  as  both  a  senior  evaluator  and  a  battalion  or 
squadron  commander  of  an  evaluated  unit.   Divisional  or 
wing  MCCRES  officers  were  also  a  rich  source  of  information 
because  of  their  daily  involvement  with  MCCRES.   The  secondary 
source  of  information  were  "others  involved  with  MCCRES." 
This  group  is  composed  of  members  of  evaluated  units  other 
than  the  unit  commanders,  such  as  unit  operations  officer, 
officers  involved  with  subunits  evaluated,  and  company 
commanders.   They  also  provided  much  information. 

Given  the  sample  identified,  the  field  study,  there- 
fore, concentrated  on  those  of  rank  from  captain  to  colonel 
in  the  military  occupational  fields  of  infantry,  aviation 
(fixed  wing  and  helicopters),  tanks,  artillery,  engineer, 
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and  air  control.  A  total  of  37  individuals  were  interviewed 
The  sample  group  is  presented  by  rank  and  occupational  field 
below   in   Table    5.1. 


TABLE  5.1 
Rank  /Occupational  Field  Sample  Distribution 


OCCUPATIONAL    £  $  a.  £ 

FIELD  §  "S  S  J2  = 


X 


c 


a> 

c 

a> 

o 

c 

(J 

o> 

c 

k. 

Q>  O  v.  C  2  Q 

RANK\^         -=:  u.  I  £  <  ui  <  P 


Colonel      3 

1     0 

0 

0     0 

0      4 

Lt.  Colonel   10 

3     3 

0 

1         1 

1      19 

Major        3 

4     3 

1 

0     0 

0      11 

Captain      2 

0     0 

1 

0     0 

0      3 

TOTAL'      18 

8     6 

2 

1         1 

'    !7 

These  officers  are 

i  currently  serving 

in  billets 

throughout 
wings,  as 

the  Marine  Corps, 

representing 

all  d 

ivisions  and 

well  as  the  1st  Marine  Brigade 

in  Hawaii.   Other 

active  duty 

officers  serving  in  billets  at 

Headq 

uarters  Marine  Corps, 

Naval  Postgraduate 

.  School,  and 

with 

the  reserve 

establishment 

were  also  interviewed. 

2.   Interview 

Techniaues 

All  those 

interviewed  were  asked  a  standard  set  of 

open-ended  questions  [Ref.  45], 

These  questions 

were  pur- 

posefully  left  brc 

ad  and  open-ended 

to  solicit  as  many  ideas 
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and  opinions  as  possible.   Each  interview  was  conducted 
in  person  or  by  telephone  by  one  of  the  researchers.   All 
interview  results  were  recorded  on  an  answer  form  and  the 
interviews  were  taped  after  the  permission  of  the  respondent 
was  given.   When  a  respondent  was  uncertain  or  uncomfortable 
in  answering  a  question,  he  was  asked  to  skip  over  it.   How- 
ever, no  attempt  was  made  to  limit  the  respondent  to  any  set 
of  prearranged  answers.   In  all  cases,  where  answer  choices 
were  posed,  an  invitation  for  "other  ideas  or  suggestions" 
was  made.   As  the  interviews  progressed,  the  questions  were 
refined  to  help  the  interviewee  narrow  in  on  the  specific 
area  being  researched. 
3 .   Questions  Posed 

The  following  set  of  questions  was  posed  to  all  those 
interviewed: 

(1)  When  the  computerized  data  results  from  a  MCCRES  are 
provided  to  the  evaluated  unit,  what  uses  are  made 
of  that  data? 

(2)  Do  you  feel  that  information/data  provided  to  the 
evaluated  unit  as  the  result  of  a  MCCRES  are  satis- 
factory for  the  purpose  of  initiating  corrective 
action  that  will  improve  combat  readiness? 

(3)  Is  the  current  procedure  of  providing  the  results  as 
a  computer  printout  based  on  the  evaluation  of  the 
Sections,  MPS,  Tasks,  and  Requirements  the  most 
effective  format  for  displaying  the  results  to  the 
evaluated  unit  commander? 
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(4)  Does  the  format  lend  itself "to  a  complete  understanding 
of  how  the  unit  performed  compared  with  MCCRES 
criteria? 

(5)  Do  Marine  Corps  Directives  provide  sufficient  guidance 
for  interpretation  of  results? 

(6)  Would  it  be  helpful  to  know  how  your  unit  performed 
on  a  Section/MPS/Task/Requirement  as  compared  with 
other  units?   Example:   Your  infantry  battalion  scored 
a  73%  on  the  MPS,  "Command  and  Control  Operations." 
This  compares  with  a  median  of  68%  for  all  other 
infantry  units  and  71%  for  all  units  evaluated. 

(7)  What  do  you  feel  is  the  most  effective  forum  for  pro- 
viding MCCRES  results  to  the  unit  tested? 

a.  Oral  debrief  at  end  of  exercise 

b.  Written  debrief  at  end  of  exercise 

c.  Progressive  oral  debrief  throughout  the  exercise 

d.  Combination  of  a-c  above 

e.  Some  other  method  than  those  suggested 

(8)  Who  should  be  made  aware  of  the  MCCRES  results? 

a.  Chain  of  command 

b.  Other  like-units  (infantry,  air,  artillery,  etc.) 

c.  All  Marine  Corps  units 
4 .   Background  Visits 

Additionally  visits  were  made  to  Headquarters  Marine 
Corps  (HQMC) ,  Readiness  Branch,  George  Washington  University 
and  to  the  Army's  7th  Infantry  Division,  Fort  Ord,  California. 
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Individuals  interviewed  during  these  visits  provided  many 
valuable  insights  into  the  development  and  overseeing  of 
MCCRES ,  as  well  as  a  detailed  understanding  of  how  the  Army 
ARTEP  system  is  employed.   Several  of  the  individuals  inter- 
viewed were  involved  in  the  early  development  of  the  MCCRES: 
Professor  W.  H.  Marlowe,  George  Washington  University;  Lt. 
Colonel  Paul  Catalone,  USMC,  and  Colonel  M.  P.  Sullivan, 
USMC  (telephone  interview) ,  and  Lt .  Colonel  R.  S.  Gibson, 
USMC  (telephone  interview) . 

E.   CONTENT  ANALYSIS/DATA  REVIEW 

Since  the  material/responses  to  the  questions  asked  were 
subjective  in  nature,  content  analysis  was  used  to  analyze 
the  results  [Ref.  46]. 

1 .   Content  Analysis  Defined 

Berelson  states:   "Systematic  content  analysis 
attempts  to  define  casual  descriptions  of  the  content,  so  as 
to  show  objectively  the  nature  and  relative  strength  of  the 
stimuli  applied  to  the  reader  or  listener"  [Ref.  46:  p.  14]. 
This  procedure  is  employed  by  constructing  an  analy- 
sis outline  which,  as  described  by  Cartwright,  embodies  the 
following  general  principles: 
Step  1  —  specify  data  needs, 
Step  2 — map  out  plans  for  tabulation, 
Step  3 — map  a  skeleton  of  the  outline, 
Step  4  —  fill  in  categories  for  each  variable, 
Step  5 — establish  procedures  for  using  the  material, 
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Step  6 — try  out  the  analysis  outline  and  use  procedure. 
[Ref.  45:  pp.  454-460] 

The  application  of  content  analysis  allows  the  re- 
searcher to  study  the  communications  and  focus  on  the  inter- 
action through  messages  which  link  communicating  parties. 
Not  all  significance  can  be  extracted  by  inspection  or  mere 
observation.   The  real  purpose  of  any  analysis  is  to  illuminate 
and  make  inferences  about  something  that  is  not  otherwise 
apparent.   Thus  the  process  of  content  analysis  is  a  process 
like  that  used  when  applying  the  congruence  definition  of 
evaluation  which  employs  objectives  and  criteria  in  develop- 
ing a  measurement  process  which  is  both  scientific  and  objec- 
tive.  The  process  is  not,  however,  totally  devoid  of  judgment 
and  does  allow  the  researcher  discretion  in  categorizing 
results  [Ref.  47] . 

2 .   Analysis  of  Specific  Responses  to  Questions 

The  purpose  of  content  analysis  is  to  provide  some 
means  of  quantifying  subjective  information.   To  accomplish 
this  rules  must  be  established.   The  data  contained  in  the 
response  to  each  of  the  eight  questions  was  treated  as  a 
separate  unit  and  no  attempt  was  made  during  the  interview  to 
relate  any  answers  from  one  question  to  that  of  another.   This 
procedure  was  used  to  solicit  as  much  information  possible 
and  not  to  limit  any  of  the  respondents  in  their  comments. 
Since  each  question  stood  alone  and  was  relatively  subjective 
in  nature,  each  question  provided  an  opportunity  for  a  response 
to  contain  similar  information  as  that  given  to  different 
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questions.   Thus  once  a  respondent  began  to  provide  informa- 
tion to  a  question  which  was  more  appropriately  categorized 
as  a  response  to  a  different  question,  the  interviewer  simply 
recorded  the  information  under  the  question  currently  being 
asked.   However  as  part  of  the  analysis  of  the  interviews, 
responses  to  one  question  that  were  relevant  to  other  ques- 
tions were  considered. 

Each  question  had  certain  categories  set  up  for  the 
responses  and  each  response  made  was  placed  in  one  of  these 
categories.   In  order  to  generate  useful  information  from 
content  analysis,  the  categories  must  be  collectively  exhaus- 
tive and  mutually  exclusive.   For  example: 

Question  #1--When  the  computerized  data  results  from  a  MCCRES 
are  provided  to  the  evaluated  unit,  what  uses  are  made  of 
that  data? 
Responses  to  this  question  fell  into  three  groupings: 

"none" 

"used  for  planning  future  training  and  resource  allocation" 

"historical  reference  only" 

It  should  be  understood  that  each  MCCRES  has  many  key 
participants  and  each  participant  has  a  different  involvement 
in  the  evaluation.   Some  such  as  the  unit  commander  and  senior 
evaluator  are  involved  virtually  in  all  aspects.   Others  such 
as  the  operations  officers  and  operations  evaluators  have  a 
more  limited  focus  in  their  responsibilities.   Thus  a  system 
was  developed  to  give  varying  weight  to  those  who  had  differ- 
ing levels  of  involvement  with  MCCRES . 
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A  scoring  system  was  established  with  the  following 
weighting: 


Intensi 

ty_ 

Points 

High 

3 

Medium 

2 

Low 

1 

High  Intensity — those  involved  directly  in  one  or  more 

MCCRES  as  unit  commander,  division  or 
wing  MCCRES  officer,  senior  evaluator. 

Medium  Intensity—those  involved  with  less  direct  responsi- 
bility for  results:   operations  officers, 
other  evaluators ,  company  commanders,  etc. 

Low  Intensity — Employed  in  only  one  case  where  interviewee 

was  involved  in  a  forerunner  evaluation  to 
MCCRES 


3 .   Analysis  of  General  Responses  to  Questions 

To  determine  which  question  responses  should  be  dis- 
played for  the  reader,  a  10%  rule  was  formulated.   If  10%  or 
more  of  the  respondents  to  a  question  amplified  their 
specific  answer  with  similar  comments  or  suggestions,  then 
the  comments  were  included.   Thus,  any  of  the  general  responses 
included  in  the  next  chapter  were  mentioned  by  four  or  more 
of  those  interviewed. 

F.   SUMMARY 

In  this  chapter  the  research  method  is  summarized.   A 
total  of  37  Marine  officers  were  interviewed.   Although  the 
number  is  relatively  small,  it  should  be  remembered  that  the 
Marine  Corps  has  three  infantry  divisions  and  three  aircraft 
wings  on  active  duty.   Therefore,  the  total  population  of 
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those  qualified  to  answer  the  questions  is  drawn  from  a 
limited  population  of  officers  in  the  target  group.   It  is 
difficult  to  estimate  the  total  number  of  officers  that  have 
been  involved  in  these  primary  capacities  since  the  billet 
holders  rotate  regularly;  but  it  is  safe  to  say  that  only  a 
few  hundred  officers  have  served  as  battalion/squadron  com- 
manders or  senior  evaluators  of  evaluated  units.   Since  the 
MCCRES  is  only  required  to  be  given  every  two  years,  the 
sample  used  appears  to  be  reasonable. 
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VI.   RESULTS 

This  chapter  presents  results  obtained  through  the  field 
study.   The  specific  and  general  responses  to  questions  posed 
were  content  analyzed.   The  basis  for  this  type  of  analysis 
is  provided  by  Berelson  [Ref .  4 6 J . 

Percentages  derived  from  the  content  analysis  are  the 
weighted  averages  of  the  points  for  each  category  as  compared 
with  total  points  for  each  question.   The  responses  are 
weighted  based  upon  the  level  of  involvement  of  the  respondent 
in  previous  MCCRES's.   In  cases  where  some  interviewees  did 
not  answer  a  certain  question,  the  percentage  is  based  only 
on  the  results  of  those  who  did  answer.   No  attempt  was  made 
to  categorize  reasons  why  those  that  failed  to  respond  did  so. 

A.   ANALYSIS  OF  SPECIFIC  RESPONSES  TO  QUESTIONS 

First  Question:   (1)   When  the  computerized  data  results 
from  a  MCCRES  are  provided  to  the  evaluated  unit,  what  uses 
are  made  of  that  data? 

Responses  to  this  question  were  grouped  into  three  cate- 
gories—None, Used  for  planning,  and  Historical  reference 
only — and  are  shown  in  Table  6.1.   The  majority  of  the  respondei: 
(63.6%)  said  that  the  results  of  MCCRES  were  used  for  planning 
future  training  or  specifically  for  future  resource  allocation. 

Second  Question:   (2)   Do  you  feel  that  information/data 
provided  to  the  evaluated  unit  as  a  result  of  MCCRES  is 
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satisfactory  for  the  purpose  of  initiating  corrective  action 
that  will  improve  combat  readiness? 

The  responses  to  this  question  are  categorized  as  posi- 
tive or  negative  and  are  shown  in  Table  6.2.   The  majority 
of  the  respondents  (81.3%)  said  that  the  information/data 
provided  from  a  MCCRES  is  satisfactory  for  initiating  correc- 
tive action  that  will  improve  combat  readiness. 

Third  Question:   (3)   Is  the  current  procedure  of  pro- 
viding the  results  as  a  computer  printout  based  on  the  results 
of  Sections ,  MPS,  Tasks,  and  Requirements,  the  most  effective 
format  for  displaying  the  results  to  the  evaluated  unit? 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.3,  64.6%  of  the 
respondents  said  that  the  current  procedure  of  providing  the 
results  as  a  computer  printout  was  not  the  most  effective 
format  for  displaying  the  results  to  the  evaluated  unit. 

Fourth  Question:   (4)   Does  the  format  lend  itself  to  a 
complete  understanding  of  how  the  unit  performed  compared  with 
MCCRES  criteria? 

The  response  to  this  question  was  categorized  as  positive 
or  negative.   As  shown  in  Table  6.4,  55.6%  of  the  respondents 
said  that  the  current  format  lends  itself  to  a  complete  under- 
standing of  how  the  unit  compared  with  MCCRES  criteria. 

Fifth  Question:   (5)   Do  Marine  Corps  Directives  provide 
sufficient  guidance  for  interpretation  of  results? 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.5,  72.9%  of  the 
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respondents  said  that  Marine  Corps  Directives  provide  suffi- 
cient guidance  for  interpretation  of  results. 

Sixth  Question:   (6)   Would  it  be  helpful  to  know  how 
your  unit  performed  on  a  Section/MPS/Task/Requirement  as 
compared  with  other  units?   Example:   Your  infantry  battalion 
scored  a  73%  on  the  MPS,  "Command  and  Control  Operations." 
This  compares  with  a  median  of  6  8%  for  all  other  infantry 
units  and  71%  for  all  units  tested. 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.6,  55.4%  of  the 
respondents  said  that  it  would  be  helpful  to  know  how  their 
unit  performed  on  a  Section/MPS/Task/Requirement  as  compared 
with  other  units. 

Seventh  Question:   (7)   What  do  you  feel  is  the  most 
effective  forum  for  providing  MCCRES  results  to  the  unit? 

This  question  was  evaluated  based  on  selection  of  one  of 
five  responses:   oral  debrief  at  end  of  exercise;  written 
debrief  at  end;  progressive  debrief;  combination  of  above;  or 
other.   As  indicated  in  Table  6.7,  all  thirty-seven  inter- 
viewees responded  to  the  question.   Of  the  total,  91.5% 
responded  that  a  combination  of  oral  and  written  debriefs 
is  the  most  effective  forum  for  providing  MCCRES  results  to 
the  unit  evaluated. 

Eighth  Question:   (8)   Who  should  be  made  aware  of  MCCRES 
results? 

The  question  was  evaluated  based  on  selection  of  one  of 
three  responses:   chain  of  command,  other  like  units,  or  all 
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TABLE  6.7 
Results  of  Question  Seven 


SONSE 


High 

#  of  Weighted 

Respondents/ Score 


INTENSITY 

Medium 

#  of  Weighted 

Respondents/ Score 


PERCENTAGES 


Low 

#  of  Weighted 

Respondents/ Score 


'C3l  debrief  at 
eJ  of  exercise* 


(0) 


(0) 


(0) 


'Mtten  at  end" 


(0) 


(2) 


(0) 


2.1 


'logressive' 


(6) 


(0) 


(0) 


6.4 


"ombmation 
cabove" 


13 


(54)  15 


(30) 


(0) 


91.5 


r\er 


(0) 


(0) 


(0) 


DTAL: 


(60) 


(32) 


(0) 


(93)    100.0 


timber  Responding:  37  Response  Rate:  100% 
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units.   As  indicated  in  Table  6.8,  60.2%  of  the  respondents 
indicated  that  all  units  should  be  made  aware  of  MCCRES 
results . 

The  responses  to  the  questions  are  summarized  for  the 
reader  in  Tables  6.9  and  6.10.   Table  6.9  shows  the  distribu- 
tion by  rank  of  the  specific  responses  to  the  eight  questions. 
Table  6.10  shows  the  distribution  by  community  (ground/air) . 
A  more  detailed  display  of  data  by  occupational  specialty 
was  available  to  the  researchers;  however,  to  protect  the 
anonymity  of  the  respondents  only  the  general  categorizations 
of  answers  is  displayed.   In  reviewing  the  data  in  Tables  6.9 
and  6.10  there  is  no  direct  evidence  of  bias  by  any  specific 
rank  or  community.   This  same  result  was  true  when  evaluating 
the  data  by  occupational  specialty. 

B.   ANALYSIS  OF  GENERAL  RESPONSES  TO  QUESTIONS 

The  primary  purpose  of  this  research  was  to  gain  insight 
and  new  ideas.   To  this  end  the  respondents  were  specifically 
asked  to  expand  upon  and  explain  their  answers .   Their 
explanations  resulted  in  the  following  information.   The  10% 
decision  rule  discussed  above  was  used  to  determine  which 
information  to  present. 

In  response  to  Question  One,  which  related  to  the  uses 
of  the  computerized  results,  the  majority  of  the  inter- 
viewees said  that  the  results  from  a  MCCRES  found  their  way 
into  unit  training  plans  and  schedules.   One  of  the  most 
serious  concerns  brought  out  in  response  to  this  question  con- 
cerned timing  [Ref.  22]  of  the  evaluation.   If  the  MCCRES  is 
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TABLE  6.8 
Results  of  Question  Eight 

NSE  INTENSITY  PERCENTAGES 

High  Medium  Low 

#  of  Weighted  #  of  Weighted  #  of  Weighted 

Respondents/Score  Respondents/Score  Respondents/Score 

T  of  5  (15)  6  (12)  0  (0)  30.7 

nand" 

Mike  1  (2)  3  (6)  0  (0)  9  1 

iits"  11  (33)  10  (20)  0  (0)  60.2 

<L:  (50)  (38)  (0)  (88)    100.0 

)er  Responding:  36  Response  Rate:  97.3% 
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TABLE  6.9 
Distribution  by  Rank  of  Responses  to  Questions 


Question  # 

Colonel 

Lt.  Colonel 

1.  a. 

None 

0 

4 

b. 

Planning 

4 

11 

c. 

Historical 

0 

3 

2.  a. 

Positive 

3 

16 

b. 

Negative 

0 

4 

3.  a. 

Positive 

1 

7 

b. 

Negative 

2 

11 

4.  a. 

Positive 

3 

8 

b. 

Negative 

0 

9 

5.  a. 

Positive 

2 

12 

b. 

Negative 

0 

5 

6.  a. 

Positive 

3 

10 

b. 

Negative 

1 

9 

7.  a. 

Oral 

0 

0 

b. 

Written 

0 

1 

c. 

Progressive 

0 

2 

d. 

Combination 

4 

17 

e. 

Other 

0 

0 

8.  a. 

Chain 

1 

7 

b. 

Like 

0 

1 

c. 

All 

3 

10 

Major 

1 
4 
4 

7 
2 

2 
6 

3 

3 

6 

1 

7 
4 

0 
0 
0 

10 
0 

2 
2 

7 


Captain 

0 
3 
0 

3 

0 

1 
0 

1 

0 
0 

1 

1 
1 

0 
0 
0 
3 
0 

1 
1 
1 


TOTAL 

5 
22 

7 

29 

6 

11 
19 

15 
12 

20 
7 

21 

15 

0 

1 

2 

34 

0 

11 

4 

21 
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TABLE  6.10 
Distribution  by  Community  of  Responses  to  Questions 


Question  # 

Ground 

Aviation 

TOTAL 

1.  a. 

None 

4 

1 

5 

b. 

Planning 

15 

7 

22 

c. 

Historical 

2 

5 

7 

2.  a. 

Positive 

21 

8 

29 

b. 

Negative 

1 

5 

5 

3.  a. 

Positive 

7 

4 

11 

b. 

Negative 

11 

3 

19 

4.  a. 

Positive 

9 

6 

15 

b. 

Negative 

6 

6 

12 

5.  a. 

Positive 

12 

8 

20 

b. 

Negative 

3 

4 

7 

6.  a. 

Positive 

12 

9 

21 

b. 

Negative 

9 

6 

15 

7.  a. 

Oral 

0 

0 

0 

b. 

Written 

1 

0 

1 

c. 

Progressive 

2 

0 

2 

d. 

Combination 

19 

15 

34 

e. 

Other 

0 

0 

0 

8.  a. 

Chain 

9 

2 

11 

b. 

Like 

2 

2 

4 

c. 

All 

11 

*0 

21 

KEY:  Ground:  Infantry,  Tanks,  Artillery,  Engineer 
Aviation:  Fixed  wing,  Helicopter,  Air  Control 
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given  sixty  days  or  more  prior  to  deployment,  then  the  unit 
may  experience  substantial  turnover  of  key  personnel  and 
nullify  the  results  of  the  MCCRES  by  changing  key  players. 
The  alternative  to  this  presents  a  serious  dilemma.   If  the 
MCCRES  is  given  too  close  to  time  of  deployment,  then  little 
time  exists  to  correct  all  but  the  most  glaring  deficiencies. 
Additionally,  since  no  "real"  second  evaluation  is  made  of 
the  shortcomings,  the  USMC  may  be  deploying  units  who  have 
not  in  fact  corrected  shortcomings  identified  during  MCCRES. 

A  second  major  problem  identified  by  the  respondents  is 
the  timeliness  with  which  the  unit  gets  the  actual  MCCRES 
report.   Some  organizations  present  at  least  a  rough  of  the 
printout  to  the  unit  at  the  time  of  the  formal  debrief,  usually 
12-24  hours  after  termination  of  the  MCCRES.   Other  units  do 
not  receive  their  formal  printout  until  there  is  a  complete 
staffing  by  division/wing,  regiment/group;  and  in  at  least  two 
cases  the  unit  did  not  receive  their  formal  feedback  until 
they  were  on  their  deployment.   One  unit  did  not  see  their 
printout  at  all  after  the  MCCRES.   (The  last  comment  is  pro- 
vided as  an  exception  to  the  10%  rule  because  of  the  serious- 
ness of  this  situation.) 

Responses  to  Question  Two  indicated  that  the  current 
MCCRES  gives  sufficient  information  to  improve  combat  readi- 
ness.  In  the  case  of  the  rotory  wing  MCCRES,  three  of  the  six 
rotory  wing  respondents  indicated  that  more  weight  was  needed 
on  flying  skills  than  is  currently  given  by  the  MPS  in  Volume 
III.   Generally  the  responses  from  the  rotory  wing  community 
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indicated  too  much  emphasis  is  given  to  non-flight  require- 
ments and  too  little  to  flight  skills.   In  addition  responses 
from  the  fixed-wing  community  indicated  that  technology  is 
changing  more  rapidly  than  the  MCCRES .   With  the  implementa- 
tion of  the  FA-18  aircraft  and  changes  to  other  aircraft,  the 
MCCRES  evaluation  needs  to  be  changed  to  be  current  and  realis- 
tic.  In  other  words  the  aircraft  can  do  a  lot  more  than 
MCCRES  requires.   Discussions  with  the  Readiness  Branch  indi- 
cate they  are  well  aware  of  the  problems  caused  by  changing 
technology  in  the  FA-18  and  other  new  aircraft,  and  are 
presently  improving  the  MCCRES  requirements  to  test  the  air- 
craft at  limits  that  more  closely  approximate  its  actual 
capabilities  rather  than  against  the  less  demanding  F-4 
Requirements . 

A  total  of  13  respondents  commented  that  MCCRES  results 
are  only  as  credible  as  the  evaluators .   The  absence  of 
standardization  of  evaluators  and  evaluator  training,  as  well 
as  the  variances  in  how  organizations  pick  evaluators  had  a 
great  impact  on  the  value  perceptions  and  the  worth  of  the 
MCCRES  as  a  measure  of  the  unit's  readiness.   This  was  ex- 
pressed by  six  of  the  fifteen  officers  of  the  aviation  com- 
munity.  Forty  percent  of  the  respondents  submitted  an 
unsolicited  comment  on  the  quality  of  the  evaluators.   If 
the  evaluator  is  not  regarded  as  highly  qualified  by  his  peers, 
then  the  evaluation  is  regarded  with  little  weight.   The 
question  of  evaluator  selection  and  bias  is  discussed  in 
more  depth  by  Conatser  [Ref.  6]  and  Wheeler  [Ref.  5], 
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A  final  problem  cited  dealt  with  the  MCCRES  concept  of  com- 
paring unit  performance  with  published  combat  doctrine.   Doc- 
trine as  codified  in  written  manuals  may  be  outdated.   If  so, 
the  USMC  is  evaluating  units  by  comparing  their  performance  to 
standards  which  would  not  be  employed  on  the  modern  battlefield 
An  example  of  this  cited  by  four  infantry  commanders  is  "maneuv 
warfare."   Maneuver  warfare  is  widely  accepted  by  Western  mili- 
tary analysts  as  the  most  promising  way  to  fight  and  win  any 
future  war  against  Soviet  tactics.   However,  maneuver  warfare 
is  not  written  into  current  Marine  Corps  doctrinal  publications 
This  shortfall  points  out  the  need  for  distribution  of  MCCRES 
results  to  all  training  and  doctrine  commands  to  ensure  they 
are  aware  of  the  strengths  and  shortfalls  that  units/individual 
display  on  MCCRES. 

The  most  common  response  to  Question  Three,  which  related 
to  effectiveness  of  the  display  of  the  results,  was  negative. 
A  percentage  of  64.6%  of  the  respondents  said  that  they  did 
not  feel  the  computerized  printout  is  the  best  way  to  dis- 
play MCCRES  results.   Nineteen  of  thirty  respondents  indicated 
more  subjective  data  was  needed  to  fully  understand  the 
units'  evaluations  and  to  be  able  to  take  corrective  actions. 
The  computer  printout  provides  a  "Yes/No"  evaluation,  but  it 
fails  to  provide  the  "why"  and  "what"  [Ref.  37] .   More  details 
are  needed  to  describe  specific  reasons  for  "No"  marks  and 
recommendations  are  necessary  to  help  the  unit  find  ways  to 
improve.   All  those  responding  negatively  (19  of  the  30 
respondents)  indicated  it  was  much  more  valuable  to  spell  out 
the  details  of  a  shortfall  and  to  suggest  corrective  action, 
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than  to  simply  provide  a  "Yes/No"  on  an  evaluation  sheet.   One 
wing  MCCRES  officer  accomplishes  this  by  writing  all  comments 
directly  on  the  MCCRES  printout  before  it  is  provided  to  the 
unit.   The  wing  MCCRES  officer  said  this  facilitates  a  much 
more  complete  understanding  of  problem  areas.   The  general 
consensus  of  the  30  respondents  to  this  question  was  that  the 
subjective,  explanatory  information  is  by  far  the  most  valua- 
ble to  the  evaluated  unit  and  the  percentile  scores  on  the 
printout  are  probably  of  greater  value  to  higher  level  head- 
quarters for  use  in  planning  future  training. 

In  response  to  Question  Four,  which  related  to  the  under- 
standability  of  the  results,  15  of  the  27  respondents  or 
55.6%  agreed  that  the  current  format  is  not  difficult  to 
understand.   In  general  those  who  indicated  a  willingness  to 
reference  the  directives  and  compare  the  printout  to  the  appro- 
priate Volume  were  satisfied  with  the  explanations  provided 
therein.   Those  who  said  the  current  format  is  difficult  to 
understand  (12  of  the  27  or  44.4%)  did  so  based  on  a  criterion 
of  the  inconvenience  of  the  system,  that  is  requiring  both 
the  printout  and  the  Volume.   The  combined  use  of  the  printout 
and  Volume  requires  a  good  bit  of  tedious  research  to  find  the 
details  of  each  MPS/Task/Requirement .   The  recent  addition  of 
a  description  of  each  MPS/Task/Requirement  on  the  MCCRES 
printout  should  help  reduce  this  problem.   On  the  other  hand, 
the  brief  description  may  cause  even  less  referencing  of  the 
source  document  to  obtain  complete  details.   One  suggestion 
was  that  key  players  from  the  unit  to  be  evaluated  should  be 
given  detailed  classes  and  instructions  on  the  MPS  several 
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weeks  prior  to  a  scheduled  MCCRES .   This  has  recently  been 
started  in  one  wing  and  the  results  are  excellent.   This 
contention  was  supported  by  both  squadron  commanders  ques- 
tioned as  well  as  by  those  in  the  Readiness  Branch  at  Head- 
quarters Marine  Corps.   Just  as  there  is  a  need  to  educate 
evaluators ,  there  is  also  a  need  to  educate  those  to  be 
evaluated  [Refs.  5,27,28], 

An  added  point  made  by  five  unit  commanders  was  that  the 
results  must  get  out  to  all  those  concerned.   If  the  response 
to  an  evaluation  is  delayed  or  the  results  simply  become  his- 
torical record,  those  who  need  to  know  at  company,  platoon, 
and  squad  level  are  never  made  aware  of  the  results.   Emphasis 
is  needed  on  informing  all  participants  down  to  the  individual 
Marine  of  the  results  of  MCCRES  [Refs.  16,39].   MCCRES  re- 
sults should  be  distributed  to  all  detachments  to  ensure 
they  know  how  their  performance  was  evaluated. 

Responses  to  Question  Five  indicated  that  the  orders 
generally  seem  well  written  and  easy  to  interpret.   The  same 
suggestion  for  education  made  on  Question  Four  applies  here. 
That  is,  any  instruction  given  to  key  unit  players  will  result 
in  better  overall  understanding  of  MCCRES  and  allows  the  unit 
to  better  use  the  results. 

Several  specific  shortcomings  in  MCCRES  were  also  noted 
in  response  to  Question  Five.   Those  relating  to  aviation  were 
addressed  under  Question  Two.   One  infantry  division  has 
found  some  serious  shortcomings,  specifically  in  the  failure 
to  provide  for  intelligence  gathering,  fire  support  coordinatic 
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and  allocation  of  tasks  from  higher  headquarters.   These  three 
areas  all  apply  to  specific  MPS  * s  that  need  expansion  or 
improvement. 

Question  Six,  which  related  to  unit  results  comparisons, 
by  far  evoked  the  most  emotional  and  intense  responses.   Any 
time  a  comparative  index  is  proposed  it  has  the  potential  for 
both  good  and  bad  results  depending  on  its  use.   Of  37  indi- 
viduals interviewed,  55.4%  favored  some  sort  of  use  of  com- 
parative scores.   Those  opposed  tended  to  do  so  based  on  the 
view  that  any  comparison  of  scores  provided  a  potential  for  use 
as  a  report  card  to  compare  units  and  specifically  unit  com- 
manders.  The  primary  objections  to  the  use  of  comparative 
scores  was  based  on  the  lack  of  overall  standardization  within 
MCCRES ,  especially  the  differences  in  evaluators ,  scenarios, 
terrain,  and  weather  conditions. 

Those  who  favored  comparative  scoring  did  so  for  various 
reasons.   Four  respondents  thought  comparative  scoring  had  value, 
but  only  for  higher  headquarters,  training,  and  doctrine  commands. 
Seven  others  thought  comparative  scoring  had  value  if  shown  to 
the  unit  in  complete  privacy.   A  final  segment  of  ten  respondents 
indicated  that  the  comparative  scoring  had  potential  to  create 
more  unit  competition.   Thus,  overall,  10  out  of  the  37  respon- 
dents supported  any  sort  of  mass  publication  of  comparative  scores 

The  results  to  Question  Seven  provided  evidence  that 
feedback  must  be  provided  in  various  forms  throughout  the 
exercise  to  be  most  effective.   As  seen  in  response  to  Question 
Three,  64.6%  wanted  more  subjective  feedback.   In  Question 
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Seven  that  was  reaffirmed.  The  respondents  said  that  by  far  th 
most  valuable  feedback  was  obtained  from  verbal,  subjective 
critiques  given  immediately  after  an  event  has  taken  place. 
A  total  of  91.5%  of  the  respondents  indicated  a  combination 
of  debriefs  and  written  results  is  most  desirable.  The  written 
results  should  not  only  include  the  objective  numerical  re- 
sults of  "Yes/No"  markings,  but  also  explanation  of  all  the 
"No/N.A."  markings. 

Question  Eight,  which  dealt  with  the  distribution  of  MCCRES 
results,  was  most  often  answered  with  "all  units,"  60.2%  of 
the  time.   The  specific  point  made  here  was  that  the  chain  of 
command  has  an  obvious  need  to  know  the  results.   All  units 
could  benefit  from  a  generic  trend  analysis  report.   Trend 
analysis  would  provide  a  cross  pollination  of  critical  infor- 
mation throughout  the  Marine  Corps  and  assist  those  involved 
in  developing  training  and  doctrine.   Those  favoring  "all 
units"  or  "other  like  units"  comprised  a  total  percentage  of 
69.3%,  provided  a  favorable  response  to  the  suggestion  of  re- 
ceiving a  generic  trend  analysis  at  regular  intervals  from 
the  MCCRES  Data  Bank.   The  fact  that  no  respondent  provided  any 
objection  to  this  idea  is  indicative  of  its  positive  value. 

C.   SUMMARY 

This  chapter  presented  the  results  of  the  field  study. 
As  can  be  seen  in  the  next  chapter  the  analysis  of  specific 
and  general  responses  to  the  eight  questions  provided  answers 
to  the  research  questions  posed  by  this  study.   In  addition 
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a  number  of  important  unsolicited  comments,  outside  the  scope 
of  this  study,  were  also  obtained.   These  comments  are  pre- 
sented in  Appendix  B. 
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VII.   DISCUSSION  AND  CONCLUSIONS 

A.   ANSWERS  TO  PRIMARY  RESEARCH  QUESTIONS 

The  results  of  the  research  have  provided  answers  to 
questions  posed  at  the  beginning  of  this  project.   The  three 
primary  research  questions  were: 

(1)  After  MCCRES ,  what  information  should  be  fed  back 
to  the  evaluated  unit  commander? 

(2)  In  what  form  should  the  evaluation  feedback  be? 

(3)  What  channel (s)  should  be  used  to  provide  the 
feedback  to  the  evaluated  unit? 

A  comprehensive  feedback  model  is  presented  which  incorporates 
the  results  for  the  three  questions. 

It  was  evident  throughout  this  study  that  the  unit  com- 
mander faces  many  constraints:   training  time,  material, 
equipment,  logistics,  and  personnel.   The  information  from 
MCCRES  should  be  provided  in  such  a  way  as  to  allow  for 
better  allocation  of  these  scarce  resources  to  the  unit. 

As  discussed  in  Chapter  II,  the  overall  goal  of  the  MCCRES 
is  not  only  to  provide  a  measure  of  unit  readiness,  but  to 
provide  effective  feedback  to  the  organization  to  enhance  the 
opportunities  to  improve  readiness  by  applying  resources  to 
areas  identified  as  deficient  by  MCCRES.   The  discussion  of 
management  control  and  evaluation,  as  presented  in  Chapter 
III,  provides  a  rich  background  for  understanding  how  to 
perform  the  MCCRES  evaluation  and  how  to  effectively  use  the 
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results  of  an  evaluation.   In  models  of  evaluation,  informa- 
tion and  feedback  play  an  important  role  in  both  performing 
and  using  the  results  of  an  evaluation.   In  general  the  models 
of  evaluation  have  common  desirable  characteristics  such  as: 
brevity,  clarity,  timeliness,  interim  products  and  reports, 
and  responsiveness  of  feedback  [Ref .  35]  . 

Additionally,  characteristics  of  the  organizational  con- 
trol system  [Ref.  44]  such  as  goals,  standards,  measurement 
system,  evaluation  and  reward  system  as  well  as  the  feedback 
loop  play  an  equally  important  role  in  the  evaluation  models 
[Refs.  32,34,35,38,  and  40]. 

Based  upon  these  findings  and  the  results  of  a  field 
study,  a  model  for  MCCRES  feedback  was  developed.   The  model 
is  presented  in  Figure  7.1.   Each  of  the  eight  major  elements 
of  the  model  that  affect  the  evaluated  unit  are  explained 
below . 


Pre-MCCRES  Brief 


Trend  Analysis 


Foilow-up  Report 


Comparative  Percentile 


Results 


Real  Time  Feedback 


Evaluator   Worksheets 


Two-way   Debrief 


Computerized    MCCRES  Printout 


Figure    7.1      Mode!  of   MCCRES  Feedback  to  an   Evaluated   Unit 
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1.  Pre-MCCRES  Briefing 

A  detailed  orientation  several  weeks  prior  to  MCCRES 
to  provide  a  firm  foundation  for  all  key  personnel.   The 
orientation  should  provide  an  explanation  of  appropriate 
Volumes  of  MCO  3501.2.   Explanation  of  the  details  of  each 
MPS  and  procedures  for  the  evaluator  allows  the  unit  to 
maximize  the  potential  from  the  MCCRES  feedback  [Ref .  37] . 

2.  Real  Time  Feedback 

Regular  verbal  feedback  throughout  the  exercise  by 
evaluators  at  each  level.   At  any  logical  break  in  the 
scenario  detailed  evaluations  of  the  most  recent  events 
should  be  provided  by  the  evaluators.   This  provides  data 
while  the  event  is  fresh  in  everyone's  minds  [Ref.  35]. 

3 .  Evaluator  Worksheets 

The  Evaluator  Worksheets  should  be  provided  directly 
to  the  evaluated  element  shortly  after  conclusion  of  the 
exercise.   This  provides  an  interim  evaluation  report  and 
allows  for  maximum  feedback  to  the  specific  element  evaluated. 
It  would  be  especially  valuable  to  subunits  and  attachments 
who  often  receive  only  minimal  feedback  under  current  proce- 
dures [Ref.  3  7]  . 

4 .  Two-way    Debrief 

At  the  end  of  exercise,  twelve  to  twenty-four  hours 
after  the  end  of  the  formal  MCCRES,  a  detailed  debrief  should 
be  held  for  evaluators  and  key  players  from  the  unit  evaluated. 
Opportunities  should  be  given  for  two-way  discussion  to  resolve 
any  confusion  or  disagreement  particularly  regarding  subjective 
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areas.   The  individuals  from  the  evaluated  unit  should  leave 
with  not  only  a  clear  understanding  of  the  deficiencies, 
but  also  with  a  clear  understanding  of  what  caused  the  "No" 
evaluations  and  suggestions  for  improvement  [Ref.  35].   Since 
the  commander  of  the  unit  evaluated  is  responsible  for  the 
employment  of  the  feedback,  he  should  have  the  flexibility 
to  tailor  the  debrief  to  his  leadership  style.   The  commander 
may  desire  to  have  only  a  few  selected  evaluators  and  officers 
present  or  the  unit  commander  may  desire  all  evaluators  and 
unit  officers  to  be  present. 

5.  Computerized  MCCRES  Printout 

The  computerized  MCCRES  printout  with  percentile 
scores  and  detailed  amplifying  remarks  should  be  provided  to 
the  unit  in  a  timely  manner  [Ref.  35].   It  does  a  unit  little 
good  to  get  the  detailed  results  when  there  is  no  time  to 
correct  deficiencies.   What  is  timely  can  vary  depending  on 
deployment  schedules,  but  getting  a  copy  of  the  results  in 
the  hands  of  the  evaluated  unit  so  that  action  can  be  taken 
is  essential.   It  is  inexcusable  and  unprofessional  to  allow 
staffing  to  delay  this  critical  information.   Delays  also 
put  the  unit  further  away  from  the  MCCRES  and  it  tends  to 
become  an  historical  document  vice  a  map  for  future  training. 

6 .  Comparative  Percentile  Results 

The  data  needed  to  compare  units  within  a  division  or 
wing  is  currently  available  with  limited  research  within  the 
division  or  wing  files.   Additionally  most  division  or  wing 
headquarters  are  well  aware  of  how  their  scores  compare  with 
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other  like  units.   Usage  of  the  general  MCCRES  data  bank  to 
provide  comparative  scores,  medians,  and  standard  deviations 
should  be  provided  to  major  headquarters  staff,  training,  and 
doctrine  commands  for  the  limited  purpose  of  planning  future 
resource  allocation  and  to  develop  training  doctrine.   These 
comparative  results  should  be  limited  to  Section/MPS/Task 
scores  and  should  not  include  any  sort  of  comparative  final 
score  for  the  units  as  a  whole.   Providing  any  sort  of  overall 
scores  that  could  be  used  to  compare  individual  units  tested 
could  well  have  a  negative,  impact  because  of  the  many  variations 
in  evaluation  conditions  that  exist.   It  is  further  recommended 
that  these  comparative  results  be  directed  only  at  division/ 
wing  level  or  higher  commands  to  reduce  the  chance  of  any 
type  of  unit  comparisons. 

An  example  of  comparative  percentile  results  that  could 
be  provided  to  division/wing  level  for  use  in  planning  future 
training  is  presented  in  Figure  7.2. 


PAST  SIX  MONTH  MCCRES  RESULTS 


Division/ 

Overall 

Wing 

Marine  Oorps 

Section 

2.0 

Operations  Performed 

93.2% 

91.1% 

MPS 

2.A 

Actions  by  Marines 

97.8% 

98.4% 

Task 

2.A.1 

Discipline 

86.4% 

90.2% 

Figure  7.2.  Sample  of  Comparative  Summary  Percentile  Results 
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7 .  Follow-up  Report 

Currently  no  follow  up  evaluation  is  made  to  confirm 
that  the  deficiencies  have  been  corrected.   Some  type  of 
final  check  is  needed.   This  check  would  best  be  accomplished 
with  little  formality.   Since  the  unit  commander  has  the 
responsibility  for  application  of  MCCRES  feedback  to  improve 
unit  readiness,  it  is  best  left  to  the  unit  commander  to  do 
the  final  check  which  certifies  that  deficiencies  have  been 
corrected.   Resource  reallocation  from  outside  the  evaluated 
unit  likewise  should  be  certified  at  the  appropriate  level. 
The  key  issue  here  is  that  the  unit  must  be  deployed  at  the 
highest  level  of  combat  readiness  and  verification  of  correc- 
tion of  MCCRES  deficiencies  at  least  on  an  informal  basis  is 
essential.   This  follow-up  should  be  made  as  simply  as  possi- 
ble.  The  unit  commander  can  appropriately  perform  this 
certification  based  on  his  own  experience  and  judgment.   A 
formal  follow-up  inspection  is  neither  necessary  nor  desirable 

8 .  Trend  Analysis 

To  gain  a  higher  level  of  across-the-board  readiness, 
a  trend  analysis  would  be  of  value  to  all  units.   The  MCCRES 
data  base  represents  a  resource  that  could  be  used  to  keep 
the  Marine  Corps  informed  of  its  overall  strengths  and 
weaknesses.   Special  emphasis  should  be  made  on  getting  these 
results  to  training  and  doctrine  organizations  throughout 
the  Marine  Corps. 
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B.   ANSWERS  TO  SECONDARY  RESEARCH  QUESTIONS 

The  results  of  the  research  provided  answers  to  the 
secondary  research  questions  posed  at  the  beginning  of  the 
project.   The  three  secondary  research  questions  were: 

(1)  What  is  the  validity  of  comparing  MCCRES  results 
given  under  different  conditions? 

(2)  Does  MCCRES  contribute  to  effective  training? 

(3)  Is  there  available  time  and  follow  up  to  ensure 
correction  of  deficiencies  identified  during  MCCRES? 

A  first  response  to  the  question  of  the  validity  of  com- 
paring MCCRES  results  given  under  differing  conditions  might 
be  to  say  that  such  comparisons  can  only  be  made  if  complete 
standardization  is  achieved.   Although  complete  standardiza- 
tion for  purposes  of  evaluation  may  seem  to  be  desirable,  this 
may  not  be  feasible.   Since  various  Marine  Corps  units  (east 
coast,  west  coast,  overseas)  are  preparing  for  different  mis- 
sions, it  is  desirable  to  simulate  their  expected  combat 
employment  through  the  use  of  different  scenarios,  terrain, 
and  even  MPS  * s .   Additionally  the  Marine  Corps  has  other  exer- 
cises, such  as  the  Combined  Arms  Exercise  (CAX's),  which  are 
used  to  evaluate  under  standard  scenario,  terrain,  and  evalua- 
tors,  but  they  have  a  more  limited  purpose  than  MCCRES. 
Additionally,  because  of  the  broader  purposes  of  MCCRES, 
complete  standardization  is  probably  not  desirable.   For  its 
intended  purpose,  MCCRES  appears  to  provide  a  relatively  high 
degree  of  objectivity  when  compared  with  other  military  systems, 
such  as  the  performance  evaluation  system  used  for  the 
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promotions  of  Marine  Officers,  and  the  results  are  likely  to 
be  useful,  at  least  for  generic  comparisons,  and  to  identify 
trends . 

In  response  to  the  question  of  MCCRES * s  contribution  to 
effective  training,  it  is  evident  from  the  data  gathered  for 
this  study  that  MCCRES  has  the  potential  to,  and  currently 
does,  improve  unit  training.   Many  units  train  to  MCCRES 
standards  and  go  so  far  as  to  use  actual  MCCRES  Section/MPS/ 
Task/Requirements  to  structure  their  unit  training.   There- 
fore as  the  Individual  Training  Standards  (ITS)  are  developed 
for  all  units,  it  is  essential  that  they  be  closely  compared 
with  MCCRES  standards  so  that  the  training  and  evaluation 
development  move  in  the  same  direction  [Refs.  5,30,31]. 

The  third  question,  time  availability  to  correct  MCCRES 
deficiencies,  has  previously  been  discussed.   No  doubt  more 
time  would  allow  for  more  training  in  deficient  areas. 
However,  the  question  of  follow  up  is  also  critical.   As 
stated  above,  at  least  an  informal  re-validation  of  deficient 
areas  is  needed  to  ensure  the  effectiveness  of  the  MCCRES. 

C.   SUMMARY 

This  research  has  provided  detailed  answers  to  the  ques- 
tions posed  at  the  beginning  of  the  study.   The  feedback  to 
the  unit  is  the  key  element  to  making  the  MCCRES  of  value  to 
the  unit  evaluated.   The  purpose  of  MCCRES  is  to  provide 
timely  and  accurate  evaluation  of  the  force  readiness.   Key 
to  improving  readiness  is  identification  of  strengths  and 
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weaknesses,  and  providing  timely  and  thorough  feedback  to  the 
evaluated  unit  and  its  chain  of  command.   The  more  timely 
and  thorough  the  feedback,  the  higher  the  likelihood  of  im- 
proved future  readiness.   The  MCCRES  is  currently  viewed  as 
a  sound,  valuable  evaluation  system.   However,  as  this  study 
has  demonstrated,  the  feedback  to  the  units  can  be  improved. 

D.   STUDY  RECOMMENDATIONS 

The  feedback  model  developed  and  displayed  as  Figure  7.1 

should  be  incorporated  as  the  minimum  acceptable  procedure 

for  providing  feedback  from  any  MCCRES.   To  accomplish  this 

the  eight  steps  in  the  feedback  model  should  be  followed 

during  each  evaluation.   The  following  information  should 

be  routinely  provided  as  feedback. 

Pre-MCCRES  Briefing — to  key  MCCRES  players  from  the 
evaluated  unit 

Real  Time  Feedback — distributed  to  all  those  evaluated 

Two-way  Debrief  at  end  of  exercise — tailored  to  needs  of 
the  unit  commander  to  ensure  full  understanding  of 
evaluation  results 

Computerized  MCCRES  Printout — provided  in  a  timely  manner 
to  allow  new  resource  allocation  prior  to  deployment 

Comparative  Percentile  Results — to  help  direct  unit  train- 
ing and  for  incorporation  by  individual  training  and 
doctrine  commands 

Follow-up  Report — done  informally  by  unit  commander  to 
ensure  closure  of  feedback  loop  by  correction  of 
identified  deficiencies 

Trend  Analysis — to  support  future  training  as  well  as 
changes  in  doctrine. 

Implementation  of  these  recommendations  would  suit  the 

stated  needs  of  fleet  units  as  well  as  serve  the  intended 
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MCCRES  purpose  of  providing  an  evaluation  of  unit  readiness 
to  higher  headquarters. 

E.   SUMMARY  OF  STUDY 

The  purpose  of  this  study  was  to  determine  the  most 
appropriate  and  effective  manner  in  which  to  provide  feedback 
to  the  unit  commander  from  a  MCCRES.   To  this  end  research  was 
conducted  into  the  areas  of  management  control  and  evaluation 
theory.   Additionally , specif ic  research  was  conducted  in 
the  area  of  existing  systems  which  measure  or  evaluate 
readiness . 

The  study  was  approached  from  a  multi-disciplinary  back- 
ground with  a  detailed  emphasis  on  the  economic  question  of 
how  better  distribution  could  be  made  of  the  scarce  resources 
of  manpower,  equipment,  and  training  time  based  on  the  re- 
sults of  MCCRES.   To  answer  the  questions  a  detailed  field 
study  was  conducted  in  which  interviews  were  completed  with 
37  Marine  officers  of  rank  from  Captain  to  Colonel  who  have 
had  involvement  with  MCCRES  as  key  billet  holders.   Addi- 
tionally much  of  the  original  documentation  that  laid  the 
groundwork  for  the  development  of  MCCRES  was  reviewed  along 
with  interviews  and  visits  with  some  of  the  original  developers 
of  the  system. 

The  results  of  the  study  are  displayed  as  an  eight  step 
feedback  model  which  is  based  on  previous  research  in  the 
fields  of  management  control  and  evaluation,  as  well  as  the 
results  of  the  field  study.   These  results  are  communicated 


113 


to  the  reader  as  a  model  in  Figure  7.1  which  is  comprised  of 
the  following  elements: 

(1)  Pre-MCCRES  Briefing 

(2)  Real  Time  Feedback 

(3)  Evaluator  worksheets 

(4)  Two-way  Debrief  at  the  end  of  Exercise 

(5)  Computerized  MCCRES  Printout 

(6)  Comparative  Percentile  Results 

(7)  Follow-up  Report 

(8)  Trend  Analysis 

The  incorporation  of  this  model  as  standard  MCCRES 
feedback  procedure  will  significantly  enhance  the  value 
of  the  results  to  the  evaluated  unit  and  will  improve  the 
understanding  of  the  resource  allocation  needs  at  all 
levels. 
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APPENDIX  A 

SAMPLE  MARINE  CORPS  COMBAT  READINESS  EVALUATION 
SYSTEM  SOFTWARE  PRINTOUT 
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.00 

4 

.97 

12 

.41 

4 

.34 

9 

.31 

0 

.00 

0 

.00 

31.03 
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2.C 


SPECIALOPS  N/A 


2.D 


EXTERSUPTD  N/A 


2.0 

INFANTRY 

2. A 

OPNPERFORM 

2. A 

.1 

ACTNSBYMAR 

TASK 

WEIGHT 

1) 

DISCIPLINE 

100.00 

2) 

DISPERSION 

91.00 

3) 

COVER 

100.00 

4) 

CAMOUFLAGE 

82.00 

5) 

SECURITY 

100.00 

6) 

RECONPATROL 

64.00 

7) 

COMBATPTRL 

55.00 

8) 

ELECTRONIC 

82.00 

9) 

NBC 

55.00 

10) 

CHEMICAL 

55.00 

11) 

ENSEMYAIR 

82.00 

12) 

POWS 

55.00 

13) 

CASUALTIES 

45.00 

MPS  TOTAL 

100.00 

BY  TASK 


EVAL 

100 
100 
100 
100 

100 

84 
100 

43 
N/A 
100 
100 

80 

80 

92 


CUM  WT 

3 

.64 

3 

.31 

3 

.64 

2 

.98 

3 

.64 

2 

.33 

2 

.00 

2 

.98 

0 

.00 

2 

.00 

2 

.98 

2 

.00 

1 

.64 

33.16 
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2.0 
2. A 
2.A.1 
2. A. 1.1 

REQUIREMENT 


1) 

SELF 

2) 

WPNS 

3) 

MAINT 

4) 

FIRE 

5) 

SUPPLY 

6) 

COMM 

7) 

NOISE 

8) 

LIGHT 

9) 

HYGENIC 

TASK  TOTAL 

2. A 

.1.2 

REQUIREMENT 

1) 

INDIVIDUAL 

2) 

UNIT 

3) 

LEADERS 

4) 

MATERIEL 

5) 

WEAPONS 

TASK  TOTAL 

INFANTRY 
OPNPERFORM 
ACTNSBYMAR 
DISCIPLINE 

WEIGHT  EVAL 

100.00  100 

80.00  100 

80.00  100 

100.00  100 

50.0  0  N/A 

100.00  100 

90.00  100 

90.00  100 

50.00  100 

100.00  100 

DISPERSION 

WEIGHT  EVAL 

100.00  100 

80.00  100 

100.00  100 

48.00  100 

76.00  100 

91.00  100 


BY  REQUIREMENT 


CUM  WT 

0.53 
0  .42 
0.42 
0  .53 
0.00 
0.53 
0.47 
0.47 
0  .26 

3.64 


CUM  WT 

0. 

,82 

0. 

,66 

0  . 

,82 

0  . 

,39 

0  , 

,62 

3.31 
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APPENDIX  B 
SUMMARY  OF  UNSOLICITED  COMMENTS 

Because  of  the  technique  employed  of  encouraging  expan- 
sion and  discussion  on  each  question,  numerous  ideas  were 
brought  forth.   Although  these  were  not  specifically  evaluated, 
they  are  of  potential  importance  and  should  be  at  least  con- 
sidered by  anyone  interested  in  MCCRES .   The  comments  identi- 
fied herein  were  selected  based  on  their  logic  and  potential 
impact  on  MCCRES.   No  attempt  was  made  to  support  or  disprove 
these  remarks;  such  an  analysis  is,  therefore,  left  for 
future  research. 

A.   PRESENT  APPROACH  TO  MCCRES 

Few  if  any  units  fail.   MCCRES  credibility  is  in  serious 
doubt  to  several  respondents  because  all  units  seem  to  pass 
with  very  similar  overall  scores.   As  a  result  of  this,  the 
final  MCCRES  grade  is  not  very  meaningful,  especially  without 
other  grades  with  which  to  compare  the  score.   As  a  result 
of  this,  two  of  those  questioned  indicated  they  saw  no  value 
at  all  to  overall  unit  grade.   They  stated  consideration  should 
be  given  to  doing  away  with  the  unit  grade  and  just  providing 
comparative-trend  analysis  scores  on  a  generic  basis.   This 
would  do  away  with  the  informal  comparison  of  scores  that  now 
takes  place  and  provide  better  support  of  the  intended  purpose 
of  not  using  the  MCCRES  to  compare  the  overall  readiness  of 
units  evaluated. 
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B.  TWO  MCCRES '  S 

The  general  feeling  among  all  the  respondents  is  that 
MCCRES  is  a  superb  opportunity  for  training  with  more  assets 
allocated  to  the  evaluated  unit  than  for  most  other  exercises 
Some  suggest  a  MCCRES  should  be  conducted  at  both  the  be- 
ginning and  end  of  a  training  cycle;  to  evaluate  improvements 
and  to  measure  the  unit  against  itself.   This  would  allow 
evaluators  to  be  more  critical,  because  the  unit  would  iden- 
tify strengths  and  weaknesses  early  and  adjust  training 
objectives  accordingly.   Although  it  is  a  costly  suggestion, 
it  has  potential  to  further  improve  readiness. 

C.  ANALYZE  INDIVIDUALS 

The  Command  needs  to  know  not  only  strengths  and  weak- 
nesses of  unit/subunit  performance,  but  the  evaluated  unit 
commander  also  needs  an  independent  assessment  of  individual 
leadership  strengths  and  weaknesses.   Providing  MPS ' s  that 
require  individual  evaluation  of  leaders  at  all  levels  would 
assist  the  unit  commanders  to  reinforce  his  own  observations 
and  to  be  more  aware  of  where  he  needs  to  place  emphasis. 

D.  TACTICAL  STERILITY 

MCCRES  rewards  these  who  follow  doctrine  explicitly  and 
avoid  innovation.   In  maneuver  warfare  innovation  and  flexi- 
bility are  the  key  to  success.   Yet  MCCRES  measures  perform- 
ance by  comparison  with  standard  doctrine.   Problems  here 
arise  not  out  of  the  MCCRES  procedure,  but  the  fact  that  much 
of  the  doctrine  upon  which  MCCRES  is  based  rapidly  is  being 
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outdated.   To  have  an  effective/believable  evaluation,  the 
whole  cycle  of  doctrine  and  inclusion  of  new  doctrines  in 
MCCRES  must  be  maintained.   The  general  thought  is  MCCRES 
rewards  those  who  "play  the  game"  and  may  punish  those  who 
are  "ahead  of  written  doctrine."   Some  respondents  said 
sufficient  strides  have  been  made  to  overcome  this  in  certain 
situations,  but  others  said  this  remains  a  widespread  system 
problem. 

E.  LEVEL  OF  APPLICATION 

Although  Reference  1  states  that  the  Fleet  Marine  Force 
Commanding  General  is  responsible  for  application  of  MCCRES, 
the  authority  is  delegated  to  various  levels.   Some  of  the 
control  is  held  tightly  at  the  division/wing  levels;  others 
delegate  the  actual  evaluation  to  regiment/group  level. 
These  differing  procedures  greatly  change  the  standardization 
of  evaluation  and  reduce  the  pool  of  potential  evaluators . 

F.  GROUPING  AND  TIMING  FOR  EVALUATION 

Most  every  MCCRES  has  a  different  grouping  of  unit  types. 
For  instance  some  organizations  evaluate  battalion-landing 
team  units  with  the  attached  artillery,  engineers,  tanks. 
Others  evaluate  only  the  infantry  battalion;  while  some  organi- 
zations use  the  MCCRES  to  evaluate  artillery  as  battalions 
and  others  do  not.   The  Marine  Corps  standard  is  that  each 
unit  be  evaluated  using  MCCRES  every  two  years,  to  ensure 
that  all  units  in  fact  are  evaluated.   A  detailed  scheduling 
system  should  be  incorporated  to  ensure  that  no  units  are 
missed  in  the  process. 
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